Ask HN: How do you correct spatial reasoning of LLMs?

5 points by mstaoru 17 days ago | 7 comments

I'm working on a complex multi-part 3D-printed product, and I needed some engineering input from LLMs. I use highest Gemini Pro reasoning levels, Fable, and K3 Max with everything tuned to highest.

The problem I'm facing is mounting a sensor on a barbell sleeve. The sensor is a semi-hollow 100mm cylinder, 60mm diameter, with a 50mm diameter 50mm deep tube cutout for the sleeve. (It's a bit more complex than that.)

I'm trying to brainstorm different clamping mechanisms, and evaluate durability and manufacturability of several designs.

No matter what I try, the output of all LLMs is absolutely wild, especially if you ask to make a technical drawing. Anthropic reliably been the worst, and Gemini, surprisingly, the best.

But none understand what goes where in any detail, besides spewing paragraph and paragraph of wild ideas including rotational cam clamps over a recessed ring TPU collet (apparently, a "brilliant pivot") and stuff like this.

Is this the "unconquered frontier"? Do I need to blow dust off my AutoCAD? :)

krapp 17 days ago |

It's wild to me that this is even a question, because of how AI brained the entire world has become. I mean, you have a tool that you know works, and a tool that you know doesn't, but you still feel the need to find a way to make the square peg fit into the round hole.

I'm sure there are ways to "correct the spatial reasoning of LLMs" (whatever that even means) but I'd question whether you even need an LLM to begin with. I know that isn't the answer you're looking for, I'm just a slack-jawed knuckle-dragging Luddite. But this seems like the kind of simple, common problem for which established solutions already existed, like "talk to an actual engineer and not a chatbot that can't even do basic math."

And if it's a matter of not wanting to pay someone, consider that if you're already paying for an inadequate simulation of knowledge, you should be willing to pay for actual knowledge.

mstaoru 17 days ago | |

I'm with you here, the same slack-jawed knuckle-dragging Luddite. I'm not happy about the state of the world, but it is what it is.

I am working on it alone and don't have access or extra money to pay an actual engineer. If I can squeeze 10% of what an actual engineer would've told me, for $20/month I'm in.

And it's not an either-or game, I can have some rough, potentially totally stupid ideas and then validate them with an engineer, instead of paying $10k to make it from scratch.

1659447091 16 days ago | | |

Agree with asking a human, that seems like a simple question for someone in that space. I'd only use the chatbot to research the web for both professional and DIY active forums, discords, and other niche online communities in the following categories: [type-of?] engineering, 3d printing, weightlifting, and body-building

Once you have that list browse through them to find which ones seem friendly to new users asking a one off engineering question to probably never being heard from again; then post your technical question about the thing and wait, probably for much longer than an LLM but if an answer comes back it's probably more useful. I included weightlifting/body building since its barbell related -- there is a level of technical detail and precision many users in those spaces have when it comes to anything workout related, including equipment and modifications.

Other than that, set your llm to gacha-game agent-mode and connect it to some cad tools (I assume they exist, not my area). Then have it iterate on multiple approaches to the specs until it meets the requirements for each of them and spilts outs paragraphs of verbose self-congratulations and a cache of files for you to review. Maybe it will reward you with the solution you are looking for the first time (w/ the added rare chance it completes all the technical work to your standard, including all of the proper working files to feed a 3d printer)

spottedmarley 17 days ago |

Everybody is working on (or waiting for) world models. Language models are not sufficient for doing work in the meatspace.

xyzzy123 17 days ago |

I would consider asking it to model its proposed designs in openscad or build123d (ideally something query-able). Then have it render and examine plausibility / suitability from different angles. Get it to render the part in use also and give instructions to think about forces and motion.

Recommend doing this in a coding harness not a chat box.

The reason I think you might have more success with this is that the model is mostly thinking about the part in words, which it can convert to a part design in CAD in code. LLMs are really good at coding. Also means it can use relative positioning and relationships.

You will be able to iterate more easily, compare things, compute properties, commit to git etc. The process is more reproducible and steerable than generative production of images.

When the LLM can look at renders of the geometry it generated, it’s easier for it to discriminate when it’s producing nonsense like misaligned parts, things that don’t fit, etc. It’s still going to kind of suck, but it will be better. The whole process of code -> render -> inspect forces the model to put up or shut up and provides grounding. Meshes > bloviating.

As far as I know today's LLMs don't have a "visual imagination" but a process like this could be a slow approximation of one. They clearly do have SOME spatial understanding (pelican tests show us that!) but it feels really non-human.

One thing missing from this is kinesthetics. Personally I am mostly not thinking in accurate visuals in mechanical design. I am imagining how the parts feel and kind of how they move and what slips first and what bends and what feels heavy. Imagining what my hands would feel. But I don't think I trust LLMs to evaluate that stuff by writing simulation code yet.