Benchmark: CadQuery vs. OpenSCAD for agentic CAD work(modelrift.com) |
Benchmark: CadQuery vs. OpenSCAD for agentic CAD work(modelrift.com) |
Yuck, this style of LLM output is awful, it’s horrible to read and does a lot to say absolutely nothing.
- "CadQuery fails loudly and early... OpenSCAD fails silently and late"
This should be obvious, from the documentation, from the way geometry construction works. OpenSCAD has no sense of failure where one shape means another won't work; you're just drawing the equivalent of 3D pixels in space. It will always potentially-meaninglessly succeed if the syntax is OK.
- Renders caught nothing that mattered
Objects with hollows are not going to show their major problems this way.
- CadQuery can be interrogated, OpenSCAD cannot
Isn't this explicit from the documentation of both? One works by iteratively building on top of a previous result that can be stored in a variable; the other doesn't.
- OpenSCAD renders have no concept of a part edge
Again — this should be explicit from the documentation, which describes no methods for operating on edges (and largely from the fact that it is declarative).
- Speed favours OpenSCAD, and it barely matters
Yes, being faster is no good if things are wrong.
- "What decides it is verifiability rather than expressiveness, and CadQuery leads there by a wider margin than the syntax difference suggests."
Yes, because the difference is semantic. Which you can get from the documentation. Building iteratively on the basis of existing geometry is inherently more verifiable, because stuff that can't work won't work.
Honestly do people not try learning CAD before they try to build an AI tool to generate CAD models? This feels like yet another situation where people who have not done the foundational conceptual work seem to think that they can avoid it and just work around it with AI.
I don't really get it. But then I think text-to-CAD is probably the strongest example of cart-before-horse thinking in the AI world as regards technical work compared to creative work.
I like build123d simply because it can export proper STEP (like CadQuery) but has a nicely python friendly design.
I’m using a single repo for all my models and Claude with build123d (plus a VS Code viz extension). Over time, I’ve built up a little folder of useful skills and I’m generally getting satisfactory results.
That said, I have had my eye on CadQuery and am looking for an excuse to try it. For those in the know: how does it compare to build123d?
The other big difference (which I have only read about) is in the way their assembly constraints work; again Build123D takes the more procedural approach (declaring "joints" etc.)
CadQuery came first; Build123D is sort of a more pythonic, robust restatement of it.
Are you willing to share your work?
Mine is available at https://jnslmk.github.io/build123d-models/?model=drill_stora... One of the things I've found more useful is the generation of a static site to view and download the models. With WASM you can make minor adjustments directly in browser
I like your approach so much more and I wish you the best of luck with it, but I think this is the kind of fundamental flaw with text-to-CAD isn't it?
CAD is show, don't tell territory. Humans can make things we can't describe, and we can extrapolate spatially from what we can describe.
after making SEVERAL observations that cadquery is pretty much better overall, they say "ehhh we're going to stay with openscad".
Cynically it always feels to me like they are written by people who not only think end users shouldn't have to learn CAD principles, they themselves shouldn't have to either.
Code complexity and repetition balloons with part complexity. Your thing will not scale well because OpenSCAD does not scale well.
Small changes will lead to subtle flaws; changes will need to be respecified in minute detail because OpenSCAD doesn't support sketch constraints that would generate correct geometry, doesn't support edges and faces that can help make parametric designs flow.
OpenSCAD is fun for making small, trivial objects, it is a good way to learn the most basic CSG operations, and it really excels at 3D representations of mathematical formulae.
But it's entirely the wrong foundation for the more general CAD-replacement thing you are trying to do.
CadQuery is not at all as bad a foundation (I think it's quite neat, and I think it could work).
But IMO much of this article talks about "findings" that should have been obvious to you if you know enough about CAD to know what it is you are replacing.
If you are continuing in this non-bRep, fast calculation approach, have you looked at generating SDF instead?
https://github.com/yuechen-li-dev/Aetheris/
It's still kinda buggy but they are actively getting fixed, but Claude and GPTs seemed to work better with my current design than other CAD stacks. Still haven't added threads yet, so can't speak for T3, but generally speaking the generated parts are pretty good quality, and I also got fillets/chamfers working so that's a plus.
https://aetheris-editable-v8.yuechenli.workers.dev/
Let me know, since I think we may be able get something cooking together.
Claude Code: Oops, I am sorry but I was wrong.
As you say it is quicker to do it by hand. Now.
I certainly wouldn’t entrust this job to a general-purpose (universal) tool that gets it 99.9 percent right.
But a domain-specific harness? Maybe it could work.
Hardware and engineering dont tolerate hallucinations, trust-me-bro numbers, deviations.
It was probably an easy model to get right, since the only "critical" dimension was the radius of the interior. But I was able to tweak the numbers on the support length & some other details relatively easily, and I had a working solution 2 hours of print time later.
(I had a CAD class in highschool, but haven't used it since).
I suspect this may be another case of "LLMs are mainly good at things you're bad at."
It's not a small change, it's from "not useful" to "pretty good"
Right, but, so what? There is very little utility here. Parts that need to be useful need to be designed to spec.
I'm the target for such material for I do use OpenSCAD to model parts I then 3D-print to fix stuff around the house and enhance the cars etc.
And yet the writing is totally insufferable AI slop:
"All six parts came out printable. Capability turned out to be the boring part of the answer. Where the two diverge is in how they fail."
Anyone who writes like this or who enjoys such kind of sentences: from the bottom of my heart, just go fuck yourself.
At a glance, not sure how you're approaching constraints, and what luck you've had with getting language models to be able to juggle them?
Using Replicad, I've successfully iterated on a 3d printed enclosure for my electronics project. It's fantastic to get everything parameter based and version controlled from revision 1.
Full disclosure, this is one of my pie-in-sky ideas that I've been letting AI loose on with only top down direction from me. I'm usually fairly hands-on even with the model in the loop, but this is an exception, because a geometry kernel is frankly beyond my ability to write solo (and I say that as someone with a reasonably strong math background). The gamble has always been that if a current model can produce "works, but crap under the hood", future models will be able to clean it up to "works, and is solid under the hood".
I'll push everything up by the weekend and drop you a line.
(Am I naïve? … Oh god, I'm naïve)
But this article is talking about functional parts. A functional part exists because of its spec, usually fails because of some combination of its material, manufacture, and spec. And the spec is chosen with that in mind.
Even working from a text description of a part is error prone because language is imprecise and interpretable. I just don't understand why AI people want to race past actual proper sensible solutions to a problem just to automate generating from text.
Just last week I had Gemini 3.8 Flash and Antigravity take an existing STL of a part and swap out a piece of it with another design (fan shrouds on a computer case front plate). In the process it created exploded parts diagrams from the STL, created 2D renders from each axis (for its own visual re-ingestion, presumably, since I neither asked for or needed them), projected measurements, and then it implemented my design swap request. Which it didn't do with SCAD, it wrote a Python program and directly manipulated the STL geometries. It went perfectly. A week before that I had GLM 5.3 and Hermes take one model that was intended to fit into a part from another designer's part and change it; I didn't really like the interior part's design so I gave both STL files to Hermes and asked for a new interior part design that could be printed with vase mode to speed it up. It did and the new part is better than the human-designed part I grabbed off the internet.
I had another task mid week where I needed a case for something but didn't have the dimensions, so I just had Gemini do deep research lookup for dimensions and to then spit out a case STL for me. It was boring but functional and a good enough starting point for me to work with.
Right now my wife is enamored with a 3D printed purse design for halloween but the separately printed hinges are a weak design (a circular mortise and tenon rotating hinge). Working with Antigravity again, it's been able to take measurements, rotate parts in space on its own and create me a new object design that's better suited to be printed in TPU. It's held dimensional accuracy and even inferred some dimensions and spacing I didn't have out of the primary purse's shape. It's gone great. Next up will be having it combine all the parts into a single STL so I can do a multimaterial print in one shot, printing the purse in PETG but the living hinge in TPU without assembly needed.
I think "it's here, now" for home and hobbyist CAD needs for functional prints. It's not displacing a mechanical engineer's job yet, but this recent iteration is markedly better than the previous generation of models at spatial reasoning and 3D design because it's suddenly capable of it at all when even 4 months ago it was not.
In your other comment you mentioned "just learn CAD", and I have already, I could have cobbled together all of this myself in Fusion 360, but that would have taken far more time than a few minutes of prompting and walking away. I know enough domain lingo to describe the task well, and if it falls flat I will do it myself, but it's coming along faster than you seem willing to accept.
Could you do meaningful things without domain language you learned the long way?
I don't doubt it is coming along. But as with all of these things, this one is painted into a corner from the very start by using OpenSCAD.
And a mechanical engineer's job isn't just fiddling away in CAD, is it? If it was I would be on the way to being one by now, and I am not.
If a non-CAD person imagines a part that they need, the chances are very high indeed that either it isn't what they need, can't be made robustly or sensibly, or if it is, it already exists in some meaningful way that can be adapted, because one of tens of millions of other people already wanted it.
This scenario — needing something very specific that can be described only in words and nobody else has ever needed before — is almost imaginary. Most stuff just isn't like that.
Now… if someone were to design an AI CAD package that could remix objects that exist and combine them, perhaps with a GUI tool that allowed a user to identify attachment points, maybe that would be interesting. But a lot of that has been solved in non-AI GUIs already.
Text-to-CAD just isn't as good an idea as it seems. You are not going to be able to magic up some novel invention from a text description without a significant amount of domain knowledge of the words needed to describe a part, and that domain knowledge usually comes from learning CAD!
(Bit like getting a coding agent to write really good code)
And simple parts, sorry, just don't need much CAD skill. Get a CAD package on a free plan, learn enough to make your thing. You will get so much more out of that and it will be easier to make changes. It'll probably have standard tools for adding threads, even.
Even image-to-CAD for a functional part requires a significant amount of understanding of what to photograph and from which angles, etc.
One of the things you get from studying CAD diagrams — like the Solidworks exercises everyone works through — is that real world parts often have surprisingly few dimensioned constraints. A lot more emerges from geometry.
So you can look at a diagram and initially think "where the heck does it define this radius!?" and find that it never needs to, because that radius emerges from tangency, parallelism, perpendicularity etc., combined with fewer measurement constraints.
Specifying it as a measurement would therefore be overconstraint; it might lead to solids that are difficult/impossible to manufacture or erroneous when another measurement is changed.
The diagram will specify the actual driving measurements that come from the part's purpose, with the intent to adapt to different real-world requirements.
Any CAD tool that works from images is going to have to approximate, which will create over-complex models with to many measurements that are more fragile with respect to changes, whereas a design that has geometrical constraints in place that are logical will be simpler and more adaptable.
With functional parts, roughly, form follows function first, and manufacturability second. But you have to understand why, to be able to make a good part. Even (perhaps especially) with 3D printing. But not uniquely. For example, would this tool be able to understand why the draft angles on a plastic part were important, what determines the bend radius of a part that was made from pressed metal, etc.?
I must stress that I am an amateur. But my amateur skills have come from reading and learning about CAD and learning what makes a viable design.
(Sorry… this was a stupidly long answer but I am hungry and I figured I should try to blurt it out before I have to get up from my desk and cook. I've edited it a lot and cut out a bit that I will probably put in another comment later)
And yes, telling it 'add draft angles to make this castable' works.
Then you are not actually getting a model with a constraint system at all. Which is the point of the thing I'm replying to.
CAD constraints are more powerful and flexible than OpenSCAD, and the power comes from analysis of what the part is actually for, not merely what it looks like.
You can't even compel OpenSCAD to keep a line and an arc tangent. If hull() can't do it, you're making it up yourself with maths.
You can use it for basic shapes and very simple geometry. But you can use Tinkercad for that too.
Should you use it for CAD generally: I think not. Because complexity balloons out of control. An AI is going to struggle as much as a human does with a complex part in OpenSCAD.
All the text-to-CAD packages are relying on it because there's lots of simple examples in the training set, and because it is declarative — there's a belief that there is a mapping from text to declarative spec and from declarative spec to an object.
But it is the wrong choice for flexible designs that adapt correctly when measurements are changed.
It's clear to me that "AI CAD" developers who think OpenSCAD is adequate to the task have a fundamental misapprehension of the capabilities of the technologies they are actually seeking to replace. OpenSCAD is not representative of the nature or capabilities of modern CAD.
I think that if you are trying to make an adaptable functional part you would be better to start from CAD drawings, with proper constraints on them, and that an AI CAD package should be interacting with those (there are some attempts to do just that).
It's just not that difficult in a modern CAD package, it's not arcane or obscure.
But if you're going to be using code-CAD, you want a programming package that can represent fundamental aspects properly. OpenSCAD cannot.
CadQuery, to be fair to the authors of this post, does now have an experimental 2D constraint solver:
https://cadquery.readthedocs.io/en/latest/sketch.html#constr...
And it has a proper bRep kernel, so it can represent vertexes, edges, arcs and faces in mathematically abstracted (not mesh approximated) forms, and do operations on those (including fillets, though it has its weaknesses there inherited from OpenCascade).
I think an AI that could operate properly on CadQuery models and implement appropriate sketch constraints with an awareness of their implications, choose the right operations, make use of libraries, etc., would be a lot more capable. But it's so much further down the line than people think based on these OpenSCAD toys that produce trivial parts.