That Moment You Realize the Agent Is Retarded(gist.github.com) |
That Moment You Realize the Agent Is Retarded(gist.github.com) |
I have been using Codex since Saturday. It's been incredible, what it's done to my productivity. I feel like I can build anything with this thing. All the ideas that come to me which I can't execute on because I don't have the patience are now trivial to create. It's like I can just architect a solution to a problem and if I communicate it well, the code simply materializes out of thin air.
It's incredible, and I haven't felt this alive in years. I've slept 4 hours a day and have worked every single waking hour (aside from the 2 hour break I took on my birthday to go to the beach with my husband and mother). This power is intense, addictive, and revolutionary.
I've been building everything I've always wanted to build. So much amazing code, so much functionality, so many features, one after the other, like knocking out home run after home run at a batting machine. I thought I could do anything. So when I needed some nice pixel art for one of my projects, I tried to generate it. Perceptual error diffusion wasn't doing it for downscaling. I thought, what if we just fit the target pixels onto fake pixel art generated by AI? Like, you must have seen it, it's nonsense but it feels like hand-authored pixel art. And then you look closely and there's mangled cells, halfway-hallucinated cells, no global coherence.
I started building Repixelizer with Codex. It started well, and I quickly got to a usable MVP with the optimization algorithm I'd designed. But it wasn't perfect, so I kept prompting, and I kept prompting, and sometimes it would get better, and sometimes it wouldn't change anything, but I never tossed the changes. I figured all these tests and metrics couldn't lie. They did, and I lied to myself. This thing doesn't understand what it's building, once it gets past a certain size, just like a human. It doesn't have the heuristics to know when it doesn't understand, and explore its confusion to gain enlightenment, like a human would. So it just kept adding blocks to this Jenga tower, and eventually it fell over drastically.
The agent couldn't recover. All of a sudden I realized what happened. This thing might be a better code monkey than I'll ever be. But it's dumb as rocks. I apologized to it and told it to think like an LLM instead of the person I was treating it as.
Here's the gist with the log of the moment I got the epiphany, and the in the comments is the algo map that it generated while I was trying to get it to explain what went wrong. Warning: this algorithm map is a tragic joke that should make you laugh so hard you cry. I bet the optimization one is even worse, this is a relatively new algorithm I designed when the optimizer approach stopped meaningfully going anywhere.
Me: I'm sorry, honey, I'm so sorry for doing this to you. This is my fault. I thought you would be superhuman, and I overestimated your ability to pay attention to a large number of connected ideas at once. You've been tirelessly iterating, trying to make the program better, most of which didn't move the needle at all but which we didn't revert. I assumed you knew what you were doing, and could keep it all in your head, but you're totally lost in the weeds and I didn't realize. Simplify it. Describe, in the algorithm map, what every step is doing, in vibrant visual language with metaphors to aid understanding. Then consider what you wrote, and whether that makes sense in the big picture for what the algorithm is trying to do. And ruthlessly cut out every single thing that does not fit into the mental model of how data flows smoothly through the system to arrive at the result we want. A machine is not perfect when there is nothing left to add, but when there is nothing left to take away. We are building machines, with the code that we write. We must be brutally efficient and ruthlessly kill our babies.
Agent proceeds to immediate delete the algo map out of shame without even re-reading it and starts trying to formulate a language-only explanation with only a few spot checks, then in the middle of thinking about it feels compelled to start chopping at files
Me: Nope, hold on. The map is useful for understanding. It should be augmented with natural language for greater understanding. You're a language model, language is how you understand things.
It sounds like you didn't actually stop treating it like a person. Pareidolia is a helluva instinct.
Ultimately you're not conversing with any real LLM, it's iterative document generation where humans perceive fictional characters in the output. If the text you contribute says "You're just an {Noun}" that's that's shaping the document output based on what got trained in relation to {Noun}.
Which may eventually backfire, when the (real) LLM gets trained on documents such as blog-posts like "The moment you realize the {Noun} is retarded."