Claude Code: Anatomy of a Misfeature(olafalders.com) |
Claude Code: Anatomy of a Misfeature(olafalders.com) |
It's Thariq from the Claude Code team here. This was my change! I made the AskUserQuestion tool so am generally in charge of maintaining it.
First, overall wanted to apologize and agree that this did not meet our bar and does not represent how we plan to ship on Claude Code.
To give you a motivating sense, as the models get more powerful, usage patterns start to change. I'd gotten a lot of feedback that AskUserQuestion tool was starting to block some long running jobs unexpectedly and so I tried a change to help that.
Our internal feedback on this was good, but the rollout should have been opt-in (like it is now) and on the Changelog.
Thanks for the feedback! We're always trying to make Claude Code better while balancing it with how people use it in many diverse ways. I did not really intend AskUserQuestion to be a safety gate when I first built it, but I realize it has evolved in that direction for some users.
I'm still exploring other ways of helping with this problem of balancing longrunning work and input, but will take lessons from the rollout here.
It's also worth adding that I really enjoy the AskUserQuestion feature and will regularly ask Claude to specifically use it instead of asking me questions in plain text because it's a lot easier to work with.
It's always good to learn from mistakes, and I appreciate both your work on this and you coming here to own it. Keep up the good work!
Wanton accountability for a multi-billion dollar cutting edge company… leaves more to be desired from the best? Take Apple or Google or any top tech company of the past at its prime and compare. This kind of behavior then would probably reflect poorly on the institutions behind the tech and not maintain their image of technical brilliance because it shows weakness in a vulnerable way. It is human. It is not strategic.
By wanton accountability, I mean things like saying “This was my change!” or “ wanted to apologize and agree that this did not meet our bar and does not represent how we plan to ship on Claude Code”. It makes them (the company) look weak? Accountability is important but where, how, and when you do it is even more important. These stakes are, not joking, life and death, and in this big game of chess we get paid for, naivety not only in our technical implementation can weaken our position.
Not trying to attack, just trying to learn and probe with the community. Maybe a cost of this kind of transparency on the internet. I am wondering if this a new trend and tech companies are changing in a way I don’t understand! In any case, it’s really cool work that is being with Claude Code!
Are you complaining that the answer is too human and that a multi billion company should not allow a human who made a mistake to own the mistake in public, being honest about what happened in this case?? Would you prefer complete silence from them like you most certainly would from Google or Apple.
That sounds incredibly sad to me, we don’t even expect humanity from big tech since that’s what we’ve become accustomed to seeing.
fTR both the Anthropic dev response and the blog post seem to believe that a single person can be blamed for something like this, which I wholeheartedly disagree with! Nobody reviews your changes? There’s no QA? Not even an AI checking the release notes match the diff from the previous release?? Blaming a dev for “putting a serious bug in production” sounds really 90’s to me.
It is such a shame that Anthropic has no interest in QA, because they have incredible models, and bafflingly broken products. In an alternate universe Claude Code would be 10x better than where it is now...
I'm using the VS Code extension over SSH.
Thanks for the openness. I got bit by this one and was, frankly, pretty surprised.
The funny thing about user-facing interaction mechanics is that everyone is part of some minority, and everyone comes with their own sense of what "natural" or "obvious" is. With something this impactful, communicated clarity of behavior will important. Your feature is also doing double-duty, serving as a last net against prompt-injection attacks by giving the user the final say.
(Also, BTW, folks outside of Anthropic are unlikely to be as tooled-up for long-running unsupervised Claude jaunts as you guys. The cost of wild success is wide adoption.)
One thing I'll suggest is that the mechanics of permissions and asking are presently pretty hacker/nerd friendly but simultaneously too-scary and not-scary-enough for non-coders.
Examples:
- Wild-cards on always approve is awesome, but, with prefixes like timeout and nohup, the "thing" that is getting done is buried and largely unexplained to the user.
- Auto is actually kind of a sweet spot (sometimes goes off into the weeds), but the designers and PM's I've been working with might as well YOLO. They have no idea if they're breaking things, but they gravitate between plan and auto mode.
- Fewer permission prompts is great, but it comes after a user has slogged through generation of a data-set to work against, like battle scars for paper cuts. It's the thermostat problem. The signal comes when the user is uncomfortable. And it's a way to learn me, but not me now.
I've had good fortune with Opus 4.8 and Fable just telling the system what phase of my life it's in. Things like "I'm going to go make dinner... Go profile the matrix or configurations and build the dataset for the next two hours while I'm away" have a pretty good hit rate. On the flip side "keep me in the loop and bring me your results before making structural changes" also articulates well with Fable. It will tread more carefully.
And these approaches are the ones we'd use with someone transitioning from SDE1 to SDE2. A little more autonomy, and the grounding in the bigger picture. Can we eventually translate to perfectly judging what the user wants in the moment based on incomplete signal?
No, but I'm glad you're trying. Keep the interaction model clear to your broad set of users, and we'll come along for the ride.
This was such a frustrating part of this incident, along with Anthropic's refusal to explain why the changelog is no longer a complete record, what else is going out? [1]
Boris Cherny's only participation in the thread was to delete "extreme danger" from the GH issue title [2]
I guess we should be thankful they added an option and disabled it by default. OpenAI is standing firm on their decision to not allow their 60s timeout to be disabled, [3] however more of the Codex harness is open source so customers have been able to fork it to add the option themselves.
[1]: https://github.com/anthropics/claude-code/issues/73125
[2]: https://github.com/anthropics/claude-code/issues/73125#event...
What i found is that you have to adapt your workflow to the specific model you are using and readapt it every time you notice some problems. Whether of opus4.8, or sonnet5.
Right now the interests align, but as soon as more tokens -> more profit (instead of more revenue and more losses) the perverse incentives will be too big to avoid.
It seems inevitable that open source harnesses will win. These companies would do better to just open source their harnesses.
Vibe-coders pumping out features: this is the reaction you're going to get if you inflict this on your users! Nobody wants software that updates every day and changes in fundamental ways that are hard to keep up with.
Copy/paste is one of the most basic, low-level features of a modern operating system. NO APPLICATION SHOULD EVER SCREW WITH IT, IN ANY WAY!
And I say this not just as a seriously annoyed user, but also as a professional UI developer: it is a well-known anti-pattern to override the user's expectations, at any level ... and that applies tenfold to the most basic patterns that every other app follows.
I don't care if you added a magic way to write all my code for me: if the only way to invoke it is to break copy/paste, you've failed at development!
Probably from trying to keep scrollback virtual while anticipating terminal resize. But I wouldn't mind a way to opt back in to naive, unlimited convo scrollback.
More reason for me to use OpenCode and my local LLMs
Furthermore, believing that the only thing saving you from disaster is Claude deciding to ask you a question is not a great conclusion either. You need guardrails in the power you bestow upon Claude from outside, not from inside.
Meanwhile, this article was written by Claude and has sentences like "Which cuts less far than it looks.", which I doubt Claude stopped to ask about.
I would love Claude Code to be a little less vibe-coded. The underlying model is excellent, but we're being pretty much forced into using CC to use the subscription model.
The most recent one that's had me annoyed is the "Fullscreen" TUI feature, which is super unintuitive, implementing its own text highlighting and copy-on-select mechanics, overriding your terminal's native right click. Easy to disable but terrible defaults, IMO. It's not even really clear to me what problem it was actually supposed to solve.
It is much worse than that. Claude Code doesn't auto-commit when stopping for an answer. There might be possible data loss if an uncommitted file is edited.
Good luck recovering the file from the JSONL conversation history.
Instead, start with a plan file and tell the agent to break it up into logical commits.
Though I think the bigger issue here is when you're yoloing something mutable, like managing a remote server or driving a browser or troubleshooting your local OS where there's no going back.
For me it sounds good.
For Anthropic it might increase load and make them less money but give them better KPIs.
I built in Human blockers into my agentic workflows with great intention, so naturally this annoyed me deeply. 14d ago, I was elated to see a random blog post that detailed the “fix” [0].
[0] https://zenn.dev/ytkdm/articles/claude-code-askuserquestion-...
There are situations when I want Claude to start working on something just as I'm about to head to bed or otherwise step away. It's kind of annoying to come back only to find that Claude worked for just 5 minutes and then decided to pause and ask a question.
That said, I think certain types of questions should not be automatable. Maybe it's already built that way, but I wouldn't want Claude to go with its recommended direction for anything related to operations like deletions, changing external systems, etc. Basically, things that cannot be undone should be a hard-block and wait for user input always.
The prose was written by me, with the research being done by Claude and also clearly attributed. I left Claude's research as a series of bullet points so that it would be clear that I'm not passing off an LLM's work as my own, but if anyone wants to dig deeper, they have some starting points to consider.
I don't publish prose written by an LLM for the same reason I would not have an LLM solve a crossword puzzle for me -- there's no joy in that.
IMO that's worse.
I cannot fathom implementing and shipping a feature to hundreds of thousands of people without even asking basic questions like: "what types of questions does Claude ask users".
Literally one of the most used plugins in their entire ecosystem, provided via their official plugin marketplace, is Superpowers. A plugin whose very first operating step is _asking numerous questions about product requirements_. Of course those prompts can't be skipped!
It wasn't even parameterized for Claude to tell the prompt what severity of question was being asked to allow at least _something_ to categorize urgency or expected response time.
Even more egregiously, 60 seconds!? The first time I noticed this happened was when it asked me a question, I turned to my second monitor to go look at some product documentation to get an answer, and by the time I turned back it had skipped me. How can I possibly provide any kind of informed answer in under 60 seconds? I can barely read some of its context for a question in 60 seconds!
I don't think they did this with malintent, but I do think this shows an enormous gap in judgement in how they handle the idea to delivery pipeline.
(I had originally replied on safety of letting LLMs skip questions but I don't think that was your point so I removed it.)
Tbf as long as that chance is low enough it doesn't matter in practice, but I have definitely seen the classifier approve things that were questionable, and I've also seen it decline things that were obviously okay.
Of course beating the code into shape for submission requires more manual work. But the draft stage is valuable to find unexpected friction points.
In a sense yes, I think it is actually reasonable to complain that the answer is too human/individualized here because it likely wasn't this individual human who made this decision, but he's making it seem like it is so that we are less likely to blame the company as a whole.
It's counterintuitive but when one singular person owns up to the problems that, at the root, are actually systemic to the decision making of the whole company it plays on the psychology of us as humans.
"I'm in charge of maintaining it" - This is not the same as "I'm in charge of all of the decision-making behind the implementation of how this tool works for users".
I actually agree exactly with your last point that one single person taking blame is counter-intuitive/non-productive here, but it actually seems like what these large companies desire is to have one person be the fall guy to play on people's sympathies.
If this were some small startup it would make sense but this is not that case.