Against vibes: When is a generative model useful(williamjbowman.com) |
Against vibes: When is a generative model useful(williamjbowman.com) |
So this is the fun one for programming.
I let AI agents do some programming on my codebases, but then I had to spend more time catching up with their changes.
So first I was bored waiting for them to finish, and then I was confused and frustrated making sense of the result.
Whereas, when I am asking AI small things like "edit this function so it does this instead", and accepting changes manually, my mental model stays synced the whole time. And I can stay active and in flow.
(Also for such fine grained tasks, small fast cheap models are actually superior because they allow realtime usage. Even small latency makes a big difference.)
It is tempting letting them loose, after they delivered unexpectedly good results for a while, but for me it is not worth it. Manually approve and actual read. (And manually edit CLAUDE.md etc. if necessary. )
Similar here. For a lot of software I write, I don't really know what the essential "abstraction" I need is until I'm actively writing it. The answers, when I get them right, look obvious in retrospect. Sometimes, starting with Claude Code, I can get there, but my mindset is that I'm using this tool to generate software that helps me immerse myself in the problem space. It's a different pace to the process - sometimes it speeds me up, sometimes I end up taking bad concepts a lot further than I normally would before getting to the better path
I will admit, that Claude has been helpful as an assistant (especially helping me with syntax I am not familiar with), but as a programmer that does things for me, it's been awful. YMMV.
Btw. a week of doing that (treating Claude as a programmer who does things for me) did help me in a way. I now have an intuitive understanding of what it means these things are not intelligence. I am now certain, that an LLM doesn't understand anything. It seems to be able to map text to some representations and then see if these representations match or compose. I know this might sound like intelligence, but in practice it's just not enough. Pattern recognition, sure. Not intelligence. Not even close.
To me it is a form of intelligence, just not general intelligence.
And yes, the trick is not treat them as intelligent, but like an idiot. Explain every single detail. Document everything in detail. Remove anything distracting. And then it might work like a charm at times.
If that means I’m actually coding instead of figuring out why xyz random plugin isn’t doing its job right now, some subsystem that I need but don’t care to learn the internals of, then I am happy.
I wanna to build a Proxmox LXC container via Ansible playbook, both things I know and use in my homelab.
It has to be 4 services running within the same container, VPN and what not. That would take me forever to find the latest and recommended:
* Each service installation proccess
* Known issues and workaround
* Firewall and what not
It sill took me 3 nights because I had to replace one of the services. I am not expert into iptables firewall, it helped me with that.
Ansible playbook was a hit and miss but it gave me the start so I fixed what was wrong and voila.
The problem is people using it for copy/paste, it works it is good enough. No understand of what is happening, security issues and alike.
I wonder why? Can the new models read mind?
> For example, I was recently trying to install a package whose name I forgot. I prompted the model to “install that x11 fake gui thing”, a trivial prompt.
Yes, they are a better search.
I would also add that there is also a subjective factor. If I enjoy writing code a lot more than reviewing it, I am going to prefer NOT using it for writing and might just use it to review.
So "hardness" is also related to how much you like/dislike doing it.
This position is untenable when from my perspective everyone writes all of their code using agents. I had to double check the year on the post to see if this was actually posted in 2026.
Despite my ethical issues with AI, I am using it for a handful of personal projects so I am at least keeping up with what the frontier models are doing and I'm quite impressed with them for doing reverse engineering (they need a lot of hand holding, but I've been able to knock out months of trial and error pretty quickly).
That being said, I'm still perplexed when people state they're getting huge gains from them in terms of knocking out boilerplate, or helping them plan out the project. I was under the impression that the former was a solved problem, and the latter was a requirement of being a decent engineer.
It want solved. There was some generic boilerplate that was added to IDEs but it wouldn't be project specific. It wasn't able to look at patterns within your own codebase and repeat them.
>and the latter was a requirement of being a decent engineer.
Most software projects are too big to fit in one engineers head. Having AI be able to research what the relevant code is, how it works, what race conditions exist, what pitfalls or other things you may run into saves a lot of time in regards to planning.
HN/twitter/etc may be something of a bubble in that regard. As far as I can tell out in the real world, most normal software developers are much more likely to be using LLMs as fancy auto-complete than to be using agents.
But because people get all bent out of shape I prefer to call it vibe coding anyway.
This is a misinformed 'critique' which always gets on my nerves, as someone who actually works with AI. The world is random. Generative models are only random in the sense that they randomly sample from the set of correct answers for a given problem (ideally). Of course LLMs make mistakes, but this has nothing to do with the fact that they are random.
Re writing code: most people find the writing of code to be a chore. For those that don’t, I don’t envy them, because that is the part that just got completely destroyed by AI. It’s becoming pretty abundantly clear that if you enjoy hand writing code that it will be a hobby rather than something you can do professionally and succeed over people who aren’t writing by hand
This reminds me of the observation that Anthropic's unsupervised LLM-generated Rust implementation of sqlite3 was correct for the subset of features they chose, but thousands of times slower (wall clock). Of course, performance will be the next skill to be targeted by expert-led RHLF, but this is a hard problem with many tradeoffs. It may prove to be time-consuming to improve.
Small models are making great strides of course, and perhaps we will soon learn to distill common sense ;) but subtlety and nuance appear physically bound to parameter count...
Do you mean they ask clarifying questions before generating a response?
this has been my story in every one of my personal projects.
I left out some expository, apologies. For example, I rely pretty heavily on using VSA as a way to keep things self contained and thus created templates for each different functionality type (background worker, crud action, etc) where they already have the logging/database connection/code flow pattern, I just need to shove in whatever business logic is needed. Now, that logic is the hard part and would require me to explain it in excruciating detail to Claude, or write it out by hand, but it take the same amount of time either way.
In the dark before times of 6 months ago, the thing would go completely off the rails and fuck it all up. In today’s world, 80% of the time it’s gonna get you pretty close to what you actually want with literally 5 words for simple tasks.
Complex tasks require more upfront work but my anecdata has demonstrated for me that complex tasks are showing similar relative reductions in upfront planning and effort to succeed
If the model understood what it sees, it wouldn't need to be treated like someone who doesn't? And if it doesn't understand, how can it be intelligent?
I just know, that if I would point a average human to a messy old codebase, he or she would just shrug helplessly. Even most programmers.
But if I tell claude to start digging in, refactor, update outdated tools .. it produces results. So there is some "understanding" I don't know how else to call it. So surely it is not a general intelligence, but it is certainly useful.
I will stick to my earlier statement (I hope I made it in this thread) - it seems to be treating blocks of texts as concepts and tries to compose those concepts like lego blocks. It is quite amazing that it can transform characters into meanings, even if it doesn't really understand these meanings, then compose them. I just don't think that's enough to call it intelligent (but certainly it's enough to find it useful for some tasks, as you point out).