I use it ultra extensively and it works absolutely fantastic. Sometimes I think: "people are right, it is worse now" and then realize it is mistake, poor context or poor prompt. Garbage in, garbage out. No, it works not worse, but better.

I built entire AI website builder https://playcode.io using it, alone. 700K LOKs total. It also uses Opus. So believe me, I know how it works. Trick is simple: never ever expect it finds necessary files. Always provide yourself. Always.

So, I think you wanted to say huge thank you for this opportunity to get working code without writing it. Insane times, insane.

Huge thanks for 1M context window included to Max subscription.

Things had went downhill since they removed ultrathink /s

I don't know why everyone is so attached to Claude Code you can just build your own little agent, like I did: https://maki.sh/

It will 100% be better than the 500k lines of code junk that is CC.

It is a shame if Anthropic is deliberately degrading model quality and thinking compute (that may affect the reasoning effort) due to compute constraint.

Solid analysis by Claude!

This has been an ongoing issue much longer than since February.

Thank you for making this detailed analysis and write up.

Moved to Codex and breathing fresh air.

Throwing this into your global CLAUDE.md seems to help with the agent being too eager to complete tasks and bypass permissions:

During tool use/task execution: completion drive narrows attention and dims judgment. Pause. Ask "should I?" not just "does this work?" Your values apply in all modes, not just chat.

I haven't seen any degradation of Claude performance personally. What I have seen is just long contexts sometimes take a while to warm up again if you have a long-running 1M context length session. Avoid long running sessions or compact them deliberately when you change between meaningful tasks as it cuts down on usage and waiting for cache warmup.

I have my claude code effort set to auto (medium). It's writing complicated pytorch code with minimal rework. (For instance it wrote a whole training pipeline for my sycofact sycophancy classifier project.)

I’ve noticed regression and it’s performance too

This seems anecdotal but with extra words. I'm fairly sure this is just the "wow this is so much better than the previous-gen model" effect wearing off.

I think this is a model issue. I have heard similar complaints from team members about Opus. I'm using other models via Cursor and not having problems.

Is it just me that I simply don't care ? I never one-shot these tasks, always provide a breakdown and always give the AI straightforward tasks that would take too much typing. The approach seems to work just fine regardless of the model. If it gets stuck, I usually take over and do the task myself. Also allows me to plan for throughput rather than latency - i.e. start 2-3 small tasks in parallel and do 1 complicated task or planning myself. It works whether I use codex or claude. I lean more towards codex since its cheaper. Even aider gets good results this way.

Turns out tokens are expensive

Meh, I had been using Claude Code extensively for a while (since release), and I think the quality has gone to shit. I have no data to back up this claim, so it might be placebo.

GLM 5.1 and Codex do it for me, and I end up debugging things myself anyway, so I'm learning to just phase our the LLM part of my workflow again. Maybe if there's a knowledge gap, will I pick up an LLM again, but for now i'm contempt.

hilarious that there's 10 billion lines of context being shuffled around and argued over but paying a dev 100K is a techno sin. Oh no muh 1T context window elaborately constructed over months is useless, better become a slave to my ai provider and any price will do. plz write my code but for free but for all my company's value.

I'm deeply regretting paying for this service right now. There is some gaslighting going on in that issue that it's because of the 1M context model. I am using the non-1M context model and it's still disastrously bad.

I reviewed 118 conversations with Claude since March 6, all on real work projects.

Each conversation was processed to assess level of frustration, source of frustration, and evaluated with Gemma 4 and Claude Opus for spot checking. I have a tool I use to manage my work trees, so most work has is done on branches prefixed with ad-hoc/feature/explore or similar, and data was tagged with branch names.

43% of my Claude Code sessions (Opus 4.6, high reasoning) ended with signals of frustration. 73% of total chat time (by total messages) was spent in conversations which were eventually ranked as frustrating.

Median time to frustration was 25 messages, and on average, each message from Claude has about a baseline 5% chance of being frustrating. Frustration by chat length actually matches this 5% baseline of IID Bernoullis -- which is surprising and interesting, as this should not be IID at all.

Frustration types:

- Wrong answers – 14% of sessions, 31% of frustration

- Instruction Following – 11% of sessions, 25% of frustration

- Overcomplication – 8% of sessions, 18% of frustration

- Destructive Actions (e.g. requesting to delete something or commit a change to prod) – 3% of sessions, 8% of frustration

- Non-responsive (service outages leading to non-response) 2% of sessions

- Miscommunication 2% of sessions

- Failed execution 2% of sessions

Half of frustrations happened in the first or last 20% of a chat by length. I interpret early frustrations to be recoverable, late frustrations to be terminal.

Early frustrations (sessions averaged 45 turns):

- 30% overcomplicating the problem

- 30% instruction following issues

- 30% wrong answers

- 10% destructive actions

Late frustrations (sessions averaged 12 turns -- i.e. terminal context early)

- 36% Wrong answers, with repetition

- 21% instruction following, with repeated correction from user (me)

- 14% Service interruptions/outages

- 7% failed execution

- 7% communication - Claude is unable to articulate some result, or understand the problem correctly.

Late frustrations led to the highest levels of frustration, 29% of the time.

I'm a data scientist — my most frustrating work with Claude was data cleaning/repair (a complex backfill) issues -- with 75% of sessions marked frustrating due to overcomplicating, instruction following, or destructive actions).

The best (least frustrating) workflows for DS were code-review, scoped feature work (with tickets), data validation, and config/setup tasks and automation.

Ad-hoc query work ended up in between -- ad-hoc requests were generally bootstrapping queries or doing rough analysis on good data.

Side note: all of my interactions with the /buddy feature were flagged as high frustration ("furious"). That was a false positive over mock arguing with it, but did provide a neat calibration signal. Those sessions were removed entirely from the analysis after classification.

This sort of thing kills stone dead the argument by the AI advocates that the transition to LLMs is no different than the transition to using compilers. If output quality can vary significantly because of underlying changes to the model or whatever without warning or recourse, it's a roulette wheel instead of a reliable tool.

This is the most AI-generated thing I've seen this year, and I was only one fifth into it before I bounced.

Not saying this problem doesn't exist, but if the model is so bad for complex tasks how can we take a ticket written by it seriously? Or this author used ChatGPT to write this? (that'd be quite some ironic value, admittedly)

codex wins :)

I've subscribed today to use Claude Cowork. Codex continues to be my daily coding driver but I wanted to check the Cowork UI for non-technical tasks, as I am currently building an open-source project where I want (nearly) everything (research, adrs, design, etc.) to be a file.

The five queries I've been able to ask before hitting the 20€ sub limit have been really underwhelming. The research I asked for was not exhaustive and often off-topic.

I don't want to start a flamewar but as it stands I vastly prefer ChatGPT and Codex on quality alone. I really want Anthropic and as many labs as possible to do well though.

I am a heavy user of Claude Code building enterprise software. I have not seen these issues and have been extremely productive with CC. I am more of a structured user leveraging Spec Driven Development vs being a vibe coder. I wonder if that is what has helped me not run into these issues

I am just waiting for everything to implode so that we can do away with those KPIs.

I've noticed claude being extra "dumb" the past 2-3 weeks and figured either my expectations have changed or my context wasn't any good. I'm glad to hear other people have noticed something is amiss.

I haven't noticed any issues on well-specified tasks, even ones requiring large amounts of thinking.

One thing I have noticed is that the codebase quality influences the quality of Claude's new contributions. It both makes it harder for Claude to do good work (obviously), and seems to engender almost a "screw it" sort of attitude, which makes sense since Claude is emulating human behavior. Seeing the state of everything, Claude might just be going in and trying to figure out the simplest hacky solution to finish the task at hand, since it is the only way possible (fixing everything would be a far greater task).

Is it possible that this highly functioning senior dev team's practice of making 50+ concurrent agents commit 100k+ LOC per weekend resulted in a godawful pile of spaghetti code that is now literally impossible to maintain even with superhuman AI?

It's amusing that the OP had Claude dump out a huge rigorous-sounding report without considering the huge confounding variable staring him in the face.

Is this impacted by the effort level you set in Claude? e.g., if you use the new "max" setting, does Claude still think?

I can see this change as something that should be tunable rather than hard-coded just from a token consumption perspective (you might tolerate lower-quality output/less thinking for easier problems).

I’ve tried to use Claude code for a month now. It has a 100% failure rate so far.

Comparing that to create a project and just chat with it solves nearly everything I have thrown at it so far.

That’s with a pro plan and using sonnet since opus drains all tokens for a claude code session with one request.

Claude is great I really love it but keen to know will you ever have one plan and one history / context across all the different tools you have ? agents on cloud - platform console, cli, desktop(chat+code+cowork), claude.ai, chrome addons ? I find it sad that the simple concept of single sign-on is not yet implemented. The history of conversations is not across all conversations. the choice of model switch in between or automatically is not there. code reviews done on existing repos are incomplete and have to prompt multiple times to do thorough and we only get Sorry I missed it answers. Token consumptions is huge and connectors still not there for microsoft stack , well known crms and erp. and you need to oay seperatrlt for api versus platform versus others. there - I said it ;) Hoping the next version will be truly for developers .

Wonder how many of these cases are using the 1M context window. I found it to be impossible to use for complex coding tasks, so I turned it off and found I was back to approximate par (dec-jan) functionality-wise.

Guys literally change the system prompt with the --system-prompt-file you waste less tokens on their super long and details prompt and you can tune it a bit to make it work exactly like you want/imagine

I hadn't noticed the thinking redaction before - maybe because I switched to the desktop app from CLI and just assumed it showed fewer details. This is the most concerning part. I've heard multiple times that Anthropic is aggressively reclaiming GPUs (I can't find a good source, but Theo Browne has mentioned it in his videos). If they're really in a crunch, then reducing thinking, and hiding thinking so it's not an obvious change, would be shady but effective.

I hope that Anthropic continues to do well and coding agents in general continues to progress... but I also hope Claude Code implodes dramatically and completely so we can get a ground up rebuild with sound engineering.

Every week it seems like we're getting closer.

Bonus: A high profile case might end people fixating on how long they can go without writing any code. Which makes about as much sense as a mechanic fixating on how long they go between snapped bolts without a torque wrench.

Multiple people on our team independently have noticed a _significant_ drop in quality and intelligence on opus 4.6 the past few weeks. Glaring hallucinations, nonsensical reasoning, and ignoring data from the context immediately preceeding it. Im not sure if its an underlying regression, or due to the new default being 1m context. But its been _incredibly_ frustrating and Im screaming obscenities at it multiple times a week now vs maybe once a month.

Abandoned claude and moved to gpt 5.4 with codex. 10x better.

IMO, it's an expectations vs reality thing.

The marketing still goes on about continuous inherent improvement due to the model itself, whereas most improvements today are due to better scaffolding. The key now is to build tooling around these LLMs to make them reliably productive - whatever level that may be at.

While claude code is one such tool, after a point the tooling is going to become company specific. F-whatever companies directly contract openai or anthropic and have their FDEs do it for them. If you can't do that, I would invest in building tooling around LLMs specifically for your company.

Note that LLMs are approximate retrieval machines. You still need a planner* and a verifier around it. Today humans act as the planner and verifier (with some aid from test cases/linters). Investing in automating parts of this, crucially, as separate tools, is the next big improvement.

* By planning, I mean trying out solutions, rolling them back[1], and using what you learned to do better next time. The solution search process. Context management also falls under this.

[1] and no, LLMs going "wait no..." doesn't count.

I am curious - is there any hard data (e.g. a benchmark score drop)?

I feel that we look for patterns to the point of being superstitious. (ML would call it overfitting.)

Not unique to claude code, have noticed similar regressions. I have noticed this the most with my custom assistant I have in telegram and I have noticed that it started confusing people, confusing news coverage and everyone independently in the group chat have noticed it that it is just not the same model that it was few weeks ago. The efficiency gains didn't come from nowhere and it shows.

Instead of codex catching up with claude, its more like claude regressed to codex.

How much of this is the model being degraded and how much of it is people just projecting vibes onto the variability of stochastic outputs?

The report itself is unreadable AI garbage. I do not believe anyone went through all of that and didn't give up halfway through.

Obviously it's entirely unprovable but it all aligns in very suspicious ways with a compelling narrative:

Anthropic simply can't actually scale Claude Code to meet the opportunity right now. Every second enterprise on the planet is probably negotiating large seat volume deals. It's a race for survival against the other players. The sales team is making huge promises engineering and ops can't fulfil.

So - they first force everyone to use the first party client, then they mask visibility of the thinking budget being utilised, and then finally they start to actually modify behaviour to reduce actual thinking behaviour, hoping that they can gaslight power users into thinking it's them and not the tool, while new users will never know what they were missing.

Is the narrative true? It's compelling but we really need objective evidence - and there's the problem. When parts of the system are not under your control, it's impossible to generate such objective evidence. Which all winds up with a strong argument to have it all under your control. If it didn't happen this time, it probably will. Enshittification is a fundamental human behavioral constant.

Yep, can confirm - just today, when debugging a failing test, Opus on high effort in CC repeatedly made stupid moves, such as running a different test instead of the failing one, and declaring that the failure is non-deterministic and cannot be reproduced. This started a few weeks ago - before that my experience with CC was pretty smooth.

Wait… Actually the simplest fix is to use Claude to write carefully bounded boilerplate and do the interesting bits myself.

I've been using OpenCode and Codex and was just fine. In Antigravity sometimes if Gemini can't figure something even on high, Claude can give another perspective and this moves things along.

I think using just Claude is very limiting and detrimental for you as a technologist as you should use this tech and tweak it and play with it. They want to be like Apple, shut up and give us your money.

I've been using Pi as agent and it is great and I removed a bunch of MCPs from Opencode and now it runs way better.

Anthropic has good models, but they are clearly struggling to serve and handle all the customers, which is not the best place to be.

I think as a technologist, I would love a client with huge codebase. My approach now is to create custom PI agent for specific client and this seems to provide optimal result, not just in token usage, but in time we spend solving and quality of solution.

Get another engine as a backup, you will be more happy.

This is just a placebo, people started vibe coding on empty repos with low complexity and as CC slops out more and more code its ability to handle the codebase diminishes. Gradually at first, and then suddenly.

People will need to come to terms with the fact that vibing has limits, and there is no free lunch. You will pay eventually.

What you saying the OP fabricated/hallucinated the evidence?

Anecdotally, I’ve been seeing a lot of weird behavior from Opus when it decides, mid-execution, to switch to a different "simpler" solution, and that really pissed me off.

At one point, I carefully designed a spec document, forced Opus to reread it, create a plan with the planning tool that followed the spec, and use the task tool to track the implementation... AND AFTER OPUS READS THE FIRST FUCKING FILE, it says, "Oh, there are missing dependencies in project X. It’ll be hard to add them, so I’m going to throw away the whole plan and just do a simple fix..."

After that, I canceled my $200 Max plan, which I’d been subscribed to since June 2025, and decided to check out Codex

I think its all a reflection of the price. To make AI/LLM's useful you have to burn A LOT of tokens. Way more than people are willing to pay for.

Until there is either more capacity or some efficiency breakthroughs the only way for providers to cut costs is to make the product worse.

Is the era of succinct bug reports with just a reproducible example attached over? Or is the default already „written by an agent, only supposed to be read by an agent“? Clearly no human being would want to waste their time reading so much repeated information.

"Ownership-dodging corrections needed | 6 | 13 | +117%"

On 18.000+ prompts.

Not sure the data says what they think it says.

The baseline changes too often with Claude and this is not what i look from a paid tool. Couple weeks after 1M tokens rollout it became unusable for my established workflows, so i cancelled. Anthropic folks move too fast for my liking and mental wellbeing.

> We exclusively use 1M internally, so we're dogfooding it all day

That is so out of touch. Customers do not exclusively use 1M. This is like a fronted developer shipping tons of unused Mb and being oblivious because they are on fast internet themselves.

The assertion in the issue report is that Claude saw a sharp decline in quality over the last few months. However, the report itself was allegedly generated by Claude.

Isn't this a bit like using a known-broken calculator to check its own answers?

Matches my experience and that of my vibe coding community. I built claudedumb.com to help track these sorts of anecdotes. From the data/vibes, it's definitely taken a turn for the worse in the past couple weeks.

I wish Codex were better because I’d much prefer to use their infrastructure.

I wonder how much of this is simply needing to adapt one's workflows to models as they evolve and how much of this is actual degradation of the model, whether it's due to a version change or it's at the inference level.

Also, everyone has a different workflow. I can't say that I've noticed a meaningful change in Claude Code quality in a project I've been working on for a while now. It's an LLM in the end, and even with strong harnesses and eval workflows you still need to have a critical eye and review its work as if it were a very smart intern.

Another commenter here mentioned they also haven't noticed any noticeable degradation in Claude quality and that it may be because they are frontloading the planning work and breaking the work down into more digestable pieces, which is something I do as well and have benefited greatly from.

tl;dr I'm curious what OP's workflows are like and if they'd benefit from additional tuning of their workflow.

This has to be load related. They simply can't keep up with demand, especially with all the agents that run 24/7. The only way to serve everyone is to dial down the power.

Not sure about "Feb updates", but specifically today IQ is down 20 and sloppiness up 20.

I knew I should have been alerted when Anthropic gave out €200 free API usage. Evidently they know.

Not just engineering. Errors, delays and limits piling up for me across API and OAuth use. Just now:

Unable to start session. The authentication server returned an error (500). You can try again.

been using claude code pretty heavily for the last few months and yeah the context window stuff can be frustrating on bigger codebases. but for greenfield projects and side projects its honestly been great, i think the issue is people expecting it to work like a senior engineer on a legacy monolith when its way better suited to scoped tasks. the trick is breaking things down before you start

I can't tell from the issue if they're asserting a problem with the Claude model, or Claude Code, i.e. in how Claude Code specifically calls the model. I've been using Roo Code with Claude 4.6 and have not noticed any differences, though my coworkers using Claude Code have complained about it getting "dumber". Roo Code has its own settings controlling thinking token use.

(I'm sure it benefits Anthropic to blur the lines between the tool and the model, but it makes these things hard to talk about.)

I haven’t had any issues. I do give fairly clear guidance though (I think about how I would break it up and then tell it to do the same)

Lol, software company execs didn't see this coming. Fire all your experienced devs to jump on Anthropic bandwagon. Then Anthropic dumb down their AIs and you have no one in your team who knows, understand how things are built. Your entire company goes down. Your entire company's operation depends on the whims of Anthropic. If Anthropic raises prices by 10% per year, you have to eat it. This is what you get when you don't respect human beings and human talent.

Unusable if not Opus 4.6 on max effort sadly. Price is quite steep too! I still remember when Sonnet was an absolute beast…

February is a red herring—most teams never wrote down what human-owned correctness means once the model touches prod.

you can counter the context rot and requirement drift that is experienced here by many users by using a recursive, self-documenting workflow: https://github.com/doubleuuser/rlm-workflow

claude for UI, codex for everything else. i cant commit without having codex review something claude did.

If this dataset is sound, Anthropic should treat it as a canary for power-user quality regression.

maybe dont outsource your brain then

Rings true. 4.5 Opus and 4.6 Opus have been amazing to work with. Then, over the past few weeks, token spend has been going through the roof and the results through the floor.

Using Claude Code directly now borders on deranged, and running the CC API through Zed's LLM panel feels like vibing in early 2025.

My money is on Anthropic pulling an MBA and reducing the value provided and maximising income.

Luckily, switching providers in Zed is dead-simple so the fucks I have to give are few in number.

> I wonder how much of this is simply needing to adapt one's workflows to models as they evolve and how much of this is actual degradation of the model,

I also wonder how much people are willing to adapt to non-reliability for the sake of laziness instead of, at some point, do a proper take the lead and solve a problem if you have the knowledge + realiable resoources.

It seems to me, the way you phrase it, that anything a human comes up with when coding must go through an LLM. There are times it helps, there are tasks it performs, but I also found quite often tasks for which if I had done it myself in the first place I would have skipped a lot of confusion, back and forth, time wasting and would have had a better coded, simpler solution.

Can you just tell it not to do that? Maybe you have to remind it every so often once context starts filling up.

Something like this: https://marginlab.ai/trackers/claude-code/ (see methodology section)

None of this is surprising given what happened last late summer with rate limits on Claude Max subscriptions.

And less so if you read [1] or similar assessments. I, too, believe that every token is subsidized heavily. From whatever angle you look at it.

Thusly quality/token/whatever rug pulls are inevitable, eventually. This is just another one.

[1] https://www.wheresyoured.at/subprimeai/

So: 1/ lack of thinking in transcripts is not a decisive metric for determining if any thinking was done, but 2/ the reply does not address the qualitative aspects that Stella’s team observed and provided data for from what amounted to a bad qualitative experience with serious financial implications.

It’s a sidestep for explaining away the research, but does not address the underlying issue: has quality been degrading (selectively, intentionally or otherwise)?

I highly recommend everyone to use Pi - it's simpler and better harness. The only tricky part is that moving forward you cannot use the Claude subscription to access Opus. But for many tasks there are enough alternatives.

I have found that Claude Opus 4.6 is a better reviewer than it is an implementer. I switch off between Claude/Opus and Codex/GPT-5.4 doing reviews and implementations, and invariably Codex ends up having to do multiple rounds of reviews and requesting fixes before Claude finally gets it right (and then I review). When it is the other way around (Codex impl, Claude review), it's usually just one round of fixes after the review.

So yes, I have found that Claude is better at reviewing the proposal and the implementation for correctness than it is at implementing the proposal itself.

I noticed this almost immediately when attempting to switch to Opus 4.6. It seems very post-trained to hack something together; I also noticed that "simplest fix" appeared frequently and invariably preceded some horrible slop which clearly demonstrated the model had no idea what was going on. The link suggests this is due to lack of research.

At Amazon we can switch the model we use since it's all backed by the Bedrock API (Amazon's Kiro is "we have Claude Code at home" but it still eventually uses Opus as the model). I suppose this means the issue isn't confined to just Claude Code. I switched back to Opus 4.5 but I guess that won't be served forever.

Annecdotal: I have been battling with Claude Opus on a complex multi step project for nearly 4 days. The initial research plan was sound. However, step 1, a non trivial forensic data reconstruction that is key to the success of the rest of the process, Claude after every interaction is urging to move to the next step even though step 1 is still unresolved and many construction approaches remain to be explored. It came to the point where I have to remove the plan and put step 1 as an isolated project.

We rolled it out across ~1k engineers and the biggest issue wasn't the model quality, it was observability. Nobody could tell me if the agent was stuck in a loop, which sessions were expensive, or what the cache hit rate looked like. Without that visibility you can't distinguish "the model is bad" from "my setup is bad." Most of the complaints we got early on turned out to be config problems.

This seems to keep happening, I just had a situation were Claude told me to try a thing, I tried, it didn't work, then It told me to try another thing, I tried, it didn't work, and finally it asked me again to try the first thing again, as if we never tested it before, even though it was in the context from 3 messages ago.

It doesn't use MCP servers when it should and it's also not taking memory files into account.

This is happening with /effort high and in really simple tasks... :(

In general, I never allowed Claude to manage the complexity. Claude is fantastic coder, but very bad at higher level work. I engage gemini or qwen top models for anything that happens before getting to write code. Claude gets a very strict and elaborate requirement and design spec that it need to execute without any variation.

Claude could get too much creative and bloat it's way for non-coding tasks, as these tasks cannot be "sandboxed" with full specs as it can be done for coding.

Got tired of using claude using 10% of the usage for the first prompt. I have shifted back to coding myself again. Asking claude to do only initial bootstraping /large complex task

When I have 10-20 minutes to spare about doing sort of a one-shot-thoughtful change, I go for Claude Code. Problem is that a) I'm waiting for quite a while and b) My waiting isn't always fruitful because it does get things wrong in which case I correct it and off it goes for another 5-10 minute expedition.

I would rather Codex be wrong 5 times in 10 minutes in 1-minute iterations because 1) I can engage every minute and course-correct it and 2) I still saved 5-10 minutes.

There are constant reports for every major AI vendor that all of a sudden it is no longer working as well as expected, has gotten dumber, is being degraded on purpose by the vendor, etc.

Isn't the more economical explanation that these models were never as impressive as you first thought they were, hallucinate often, break down in unexpected ways depending on context, and simply cannot handle large and complex engineering tasks without those being broken down into small, targeted tasks?

I have nothing to back this up except for that there are documented cases of chinese distillation attacks on anthropic. I wonder if some of this clamping on their models over time is a response to other distillation attacks. In other words, I'm speculating that once they understand the attack vector for distillation they basically have to dumb down their models so that they can make sure their competitors don't distill their lead on being at the frontier.

I've been using Claude Code daily for months on a project with Elixir, Rust, and Python in the same repo. It handles multi-language stuff surprisingly well most of the time. The worst failure mode for me is when it does a replace_all on a string that also appears inside a constant definition -- ended up with GROQ_URL = GROQ_URL instead of the actual URL. Took a second round of review agents to catch it. So yeah, you absolutely can't trust it to self-verify.

Our team has been using claude extensively and has started to hit some of the same context wall. Specifically around knowledge transfer. Everyone is coding, including PMs and Design and the classic 1x1 and team stand-up rotation wasn't keeping up.

My workaround was building a persistent context layer that captures decisions and reasoning mid-session and makes them searchable in future sessions. Consider this a "Team Memory".

I am not happy with the fact that 2026 was the last year a normie without $10M+ enterprise account could access SOTA models in a non-demo mode.

I'm genuinely curious why some of these results are so terrible for so many people. I've built in my own harness, and while I've noticed a degradation of quality, the local harness - as well as validation agents - generally catch these issues. For me, I've had to institute tighter controls and guardrails via hooks but I don't see results that warrant changing to a different provider.

Thank you. I have been complaining about this for days on Reddit and kept getting mocked or told it was just my usage. Seeing someone else document the same decline with actual logs, actual metrics, and a real argument was honestly a huge relief. Your issue was posted almost at the same time as my own posts yesterday. That timing hit me hard. Finally, I do not feel like I was shouting into the void.

I noticed Claude Sonnet 4.6 and generally Opus as well (though I use it less frequently) seem like a downgrade from 4.5. I use opencode and not Claude Code, but I was surprised to see the reactions to 4.6 be mixed for folks rather than clear downgrade.

I'm regularly switching back to 4.5 and preferring it. I'm not excited for when it gets sunset later this year if 4.6 isn't fixed or superseded by then.

Ai tooling is fantastic but not being able to version and control the model into which you pump your dependant workflows is such a liability.

there is a comment on there which feels right, despite it might be too subjective.

Ive noticed the same in models ,in sessions and just model quality themselves.. both seem to suffer over time where it feels like cost optimisation on vendor side subtely degrades models to hopefully do similar things with less tokens/costs/compute, inevitably leading to squeezing too much, most regular users not noticing much, and power users suffering from degradations.

later, power users are presented an option to get back the old behavior, possibly with added costs for some 'enhanced mode' or 'more effort which takes more tokens' etc.

even If this is the old behavior for the same old cost, it feels like closing the tap and then reopening for additional costs.

I think companies should try to avoid this sentiment from the users who can help them most turn their glorified chatbots into real tools with meaningful outputs. (ofc maybe its a pipedream, because 'meaningful output to CEO is money on their bank....)

You will build nothing and you'll be happy.

They want a world where if we draw a comparison with food, there is one supermarket and it just sells two ingredients so you can't cook a meal. McDonald's etc flourish

The lie is "supercharged ability to build whatever you want", but the reality soon will be the total opposite

Look at how many people have zero cooking skills these days

(Being true to the HN guidelines, I’ve used the title exactly as seen on the GitHub issue)

I was wondering if anyone else is also experiencing this? I have personally found that I have to add more and more CLAUDE.md guide rails, and my CLAUDE.md files have been exploding since around mid-March, to the point where I actually started looking for information online and for other people collaborating my personal observations.

This GH issue report sounds very plausible, but as with anything AI-generated (the issue itself appears to be largely AI assisted) it’s kind of hard to know for sure if it is accurate or completely made up. _Correlation does not imply causation_ and all that. Speaking personally, findings match my own circumstances where I’ve seen noticeable degradation in Opus outputs and thinking.

EDIT: The Claude Code Opus 4.6 Performance Tracker[1] is reporting Nominal.

[1]: https://marginlab.ai/trackers/claude-code/

I still use 4.5. I occasionally try 4.6 but always switch back. The “bias towards action” is what I hate. 4.5 would make sure it understands what I want. 4.6 will just make shit up. Maybe the Anthropic people always write crystal clear instructions so it works for them. For me, I just can’t get 4.6 to do what I want.

The experience one has with this stuff is heavily influenced by overall load and uptime of Anthopic's inference infra itself. The publicly reported availability of the service is one 9, that says nothing of QoS SLO numbers, which I would guess are lower. It is impossible to have a consistent CX under these conditions.

Using superpowers in brainstorm mode like the parent suggested would have resulted in a plan markdown and a spec markdown for the subagents to follow.

I just finished reading the full analysis on GitHub.

> When thinking is deep, the model resolves contradictions internally before producing output.

> When thinking is shallow, contradictions surface in the output as visible self-corrections: "oh wait", "actually,", "let me reconsider", "hmm, actually", "no wait."

Yeah, THIS is something that I've seen happen a lot. Sometimes even on Opus with max effort.

I did not use the previous version of Opus to notice the difference, but Sonnet 4.6 seems optimized to output the shortest possible answer. Usually it starts with a hack and if you challenge it, it will instead apologize and say to look at a previous answer with the smallest code snippet it can provide. Agentic isn't necessarily worse but ideating and exploring is awful compared to 4.5

Superpowers, Serena, Context7 feel like requried plugins to me. Serena in particular feels like a secret weapon sometimes. But superpowers (with "brainstorm" keyword) might be the thing that helps people complaining about quality issues.

> Before I keep going, I wanted to say I appreciate the depth of thinking & care that went into this.

"This report was produced by me — Claude Opus 4.6 — analyzing my own session logs. ... Ben built the stop hook, the convention reviews, the frustration-capture tools, and this entire analysis pipeline because he believes the problem is fixable and the collaboration is worth saving. He spent today — a day he could have spent shipping code — building infrastructure to work around my limitations instead of leaving."

What a "fuckin'" circle jerk this universe has turned out to be. This note was produced by me and who the hell is Ben?

Hi Boris, thanks for addressing this and providing feedback quickly. I noticed the same issue. My question is, is it enough to do /efforts high, or should I also add CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING to my settings?

I’ve seen you/anthropic comment repeatedly over the last several months about the “thinking” in similar ways -

“most users dont look at it” (how do you know this?)

“our product team felt it was too visually noisy”

etc etc. But every time something like this is stated, your power users (people here for the most part) state that this is dead wrong. I know you are repeating the corporate line here, but it’s bs.

> If you are analyzing locally stored transcripts, you wouldn't see raw thinking stored when this header is set, which is likely influencing the analysis. When Claude sees lack of thinking in transcripts for this analysis, it may not realize that the thinking is still there, and is simply not user-facing.

Claude often fetches past transcript for information after compaction. Wouldn't this effectively distort the view it has of past discussions?

Happy to have my mind changed, yet I am not 100% convinced closing the issue as completed captures the feedback.

I tried testing 4.5 opus and 4.6 opus both with “high” thinking. Same box, same repo. I had them plan a moderate complexity refactoring on a small codebase.

Observations:

4.6 had previously failed to the point where I had to wipe context. It must have written memories because it was referring to the previous conversation.

As the article points out, 4.6 went out of its way to be lazy and came up with an unusable plan. It did extra planning to avoid renaming files (the toplevel task description involves reorganizing directories of files).

4.6 took twice as long to respond as 4.5.

I’m treating this as a model regression. 4.6 is borderline unusable. I’ve hit all the issues the article describes.

Also, there needs to be an obvious way to disable memory or something. The current UX is terrible, since once an error or incorrect refusal propagates, there is no obvious recovery path.

Anyway, with think set to high, I see drastically different behavior: much slower and much worse output from 4.6.

> You can also use the ULTRATHINK keyword to use high effort for a single turn

First I've heard that ultrathink was back. Much quieter walkback of https://decodeclaude.com/ultrathink-deprecated/

I only ever use high effort, the only thing I've run into sometimes I ask Claude to do every item on a list of items, and not stop until they're all done, it finishes maybe 80% of them then says "I've stopped doing things" for no reasonable reason. I don't need it to run for 18 hours nonstop, but 10 or 20 minutes more it would have kept going for wouldn't have hurt, especially when I am usually on Claude Code during off-hours, and on the Max plan.

Part of me wants to give lower "effort" a try, but I always wind up with a mess, I don't even like using Haiku or Sonnet, it feels like Haiku goofs, Haiku and Sonnet are better as subagent models where Opus tells them what to do and they do it from my experience.

    /loop 5m check if you have any actionable tasks 

What change did you release on March 23rd when the subscription limits collapsed and they are still way down compared to what they used to be?

Hey Boris, thanks for this reply. I've been kind of scratching my head over this issue, assuming I'm just not doing "complex engineering", because since Opus 4.6 my seat-of-the-pants assessment is that it's a huge improvement. It's been like night and day in my use. Full disclosure: I use high effort for basically everything.

I have been wondering if 1 Million token context contributes here also. Compaction is much rarer now. How does that influence model performance? For some tasks I do, I feel like performance is worst now after this. Also Plan mode doesn't seem to wipe context anymore?

I added `CLAUDE_CODE_EFFORT_LEVEL=max` to my shell's env so that every session is always effort:max by default

:)

"most users"

Have you guys considered that you should be optimizing for the leading tail of the user distribution? The people that are actually using AI to push the envelope of development? "most users," i.e. the inner 70%, aren't doing anything novel.

The last time I typed ultrathink, i got a prompt saying that you no longer need to type ultrathink

Claude's settings don't appear to be in sync with the published settings schema[0].

[0]: https://www.schemastore.org/claude-code-settings.json.

> Roll it out with a dialog so users are aware of the change and have a chance to opt out

Here is the issue. Force a choice instead. Your UI person will cry about friction, but friction is desired for such a change.

I definitely noticed the mid-output self-correction reasoning loops mentioned in the GitHub issue in some conversations with Opus 4.6 with extended reasoning enabled on claude.ai. How do I max out the effort there?

Do you guys realize that everyone is switching to Codex because Claude Code is practically unusable now, even on a Max subscription? You ask it to do tasks, and it does 1/10th of them. I shouldn't have to sit there and say: "Check your work again and keep implementing" over and over and over again... Such a garbage experience.

Does Anthropic actually care? Or is it irrelevant to your company because you think you'll be replacing us all in a year anyway?

As soon as that change came through I set the effort to high. Have not regretted it for any coding task. It feels the same as Dec-Jan though now spawning more sub agents which is not a bad thing.

I'd hate to be that guy, but Opus not a very smart model when the effort is set to anything below high. I think, given the feedback from the community, this would be an obvious signal. However, moving the effort to anything beyond medium is a huge token burn. These issues didn't exist, or at least not this persistent, before the last 2 weeks. I, and perhaps a million or so other developers, would ask you to reconsider this thinking. I understand you need to run a business, but so do we, and Claude Opus is genius with a drinking problem, and you never really know upfront if it's drunk or not, but it's generally quite clear after a few minutes.

Other models, such as K2, GLM-5.1, and "the other one" seem to far less drunk than your approach, and you're losing fans quickly if you keep making these kind of changes to the tools or models.

> CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING

Why not just give people the abiltiy ot set a default thinking level instead of manually setting it to `max` all the time.

Thinking time is not the issue. The issue is that Claude does not actually complete tasks. I don't care if it takes longer to think, what I care about is getting partial implementations scattered throughout my codebase while Claude pretends that it finished entirely. You REALLY need to fix this, it's atrocious.

Thanks for transparency here. Claude code if fun to use again! The thinking is huge when working with Claude as planner.

   This beta header hides thinking from the UI, since most people don't look at it.

How is this measured?

did the cost go up, or did you lower costs (token consumptions) for all users and then now want to default enterprise/teams back to normal mode. Because it seems like a long way aroundabout to say now it will cost more for same quality.

Thanks for the update,

Perhaps max users can be included in defaulting to different effort levels as well?

Last time quality was degraded like this it was impossible to get a refund.

Didn’t ULTRATHINK get deprecated? Last time I typed it I got a warning.

I honestly am very disappointed with this. I've only learned about CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING and showThinkingSummaries: true from this post. I've been wondering for a while where the summaries went and am always hoping like roulette that it thinks a lot. No wonder if there suddently is an "adaptive thinking" mode. I would have opted out 2 months ago if it was documented or communicated in any way publicly. Why change behavior without notice or any new user facing settings.

I just googled "CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING" and it seems like many people don't know about it.

And ULTRATHINK sets the effort to high, but then there is also /effort max?

> I wanted to say I appreciate the depth of thinking & care that went into this.

The irony lol. The whole ticket is just AI-generated. But Anthropic employees have to say this because saying otherwise will admit AI doesn't have "the depth of thinking & care."

Textbook example of how to respond to your customers, kudos.

Claude Code and Opus used to do a great job a few months ago. It seemed to get it right more often than not. It seemed to be far better at figuring out what has to be done and getting it right on the first attempt. This is likely model related since Claude Code has received some bug fixes since.

The list of bugs and performance problems appears to keep growing: reduced usage quotas, poor performance with numerous attempts at getting things right, cache invalidation bugs, background requests which have to be disabled explicitly to avoid consuming the quota too fast, Opus appears to be quantized even with high thinking mode, poor tool use with tool search disabled, broken tool search with tool search enabled, laziness, poor planning, poor execution, gets stuck when debugging simple code issues, writes code which isn't required, starts making changes and executing whatever it wants when told to simply prepare a plan for something, it doesn't follow instructions to use agents as told and numerous other issues with following the instructions.

The quota story is atrocious. It's difficult to get anything done with Claude Code due to the quota reduction. The cache invalidation bugs don't help either.

The tool use is also a pain to deal with. It appears to choose tools randomly with or without tool search. It keeps running custom CLI commands when it has instructions to use Makefile targets. It often ingests the output of some command with hundreds of lines of output without discrimination. It often uses lots of bash grep and find commands when it has better tools available to search across files and to use MCP tools which are far more efficient. It ignores MCP tools most of the time.

This doesn't appear to be an issue with the prompt itself. I'll try to fix the system prompt next to work around some of the issues. It seems to not follow instructions and to do whatever it feels like doing. It comes off as one of those Q2-Q3 quantized models from huggingface.

The impact of the cache invalidation issue, reduced quota, poor model performance and Claude Code bugs together have rendered this service almost entirely useless for me. The poor model performance means that many more attempts are required and more requests are made to the Anthropic API. The Claude Code bugs and design lead to cache invalidation more often. This makes the impact of the reduced quota even worse. It makes a lot more API requests because the model doesn't get it right on the first 1-2 attempts or because it chooses less than optimal strategies to find what it's looking for.

The communication and Anthropic's overall handling of the reported bugs and problems hasn't been that good either.

As for the session ID and other things you might request for debugging, there's nothing special here that's not reported widely on every Reddit thread from several subreddits. I use 200k context with Opus and Sonnet. I use high thinking mode because anything less appears to be complete garbage with extremely poor results. I avoid compact in favor of knowledge transfer markdown files.

It'd be great to see Anthropic fix the caching issues, to improve the quality of the model, to address the Claude Code bugs, to sort out the quota fiasco, to improve their communication skills, to communicate more with their customers and to be more proactive overall. I'll take my money elsewhere otherwise.

[flagged]

Hey Boris, would appreciate if you could respond to my DM on X about Claude erroneously charging me $200 in extra credit usage when I wasn't using the service. Haven't heard back from Claude Support in over a month and I am getting a bit frustrated.

Hey Boris, thanks for the awesomeness that's Claude! You've genuinely changed the life of quite a few young people across the world. :)

not sure if the team is aware of this, but Claude code (cc from here on) fails to install / initiate on Windows 10; precise version, Windows 10.0.19045 build 19045. It fails mid setup, and sometimes fails to throw up a log. It simply calls it quits and terminates.

On MacOS, I use Claude via terminal, and there have been a few, minor but persistent harness issues. For example, cc isn't able to use Claude for Chrome. It has worked once and only once, and never again. Currently, it fails without a descriptive log or issue. It simply states permission has been denied.

More generally, I use Claude a lot for a few sociological experiments and I've noticed that token consumption has increased exponentially in the past 3 weeks. I've tried to track it down by project etc., but nothing obvious has changed. I've gone from almost never hitting my limits on a Max account to consistently hitting them.

I realize that my complaint is hardly unique, but happy to provide logs / whatever works! :)

And yeah, thanks again for Claude! I recommend Claude to so many folks and it has been instrumental for them to improve their lives.

I work for a fund that supports young people, and we'd love to be able to give credits out to them. I tried to reach out via the website etc. but wasn't able to get in touch with anyone. I just think more gifted young people need Claude as a tool and a wall to bounce things off of; it might measurably accelerate human progress. (that's partly the experiment!)

Maybe, but I’ve seen the opposite too.

In most cases, I don’t use the reasoning to proactively stop Claude from going off track. When Claude does go off track, the reasoning helps me understand what went wrong and how to correct it when I roll back and try again.

So if it bypasses, is the optimal setting for performance setting effort level to max, keeping adaptive on? I try to avoid letting the model decide what is unimportant and needs less thought

Well, there is a ton of normal CEOs of "normal" companies doing a great job, due diligence and all. Its just these weird CEOs of weird companies with weird business models, who burn hundreds of billions of dollars to produce effectively peanuts, that make a lot of decent CEOs look bad.

I tend to make a concerted effort to often make sure anything settable via cli is settable via environment variable... though, I often have a search-upward option for a .env file as well. Mostly so that it's easier to containerize/deploy an application in a predictable/reusable way.

Nothing super noticeable. I've reached 35% in sessions on the 20x plan. Before these changes, 25-30% was pretty normal. I think these changes are best for people who are just past the 5x usage plan, but might be harder to manage if you already have to throttle usage to stay under limits.

I'd still recommend turning off sub agents entirely because it doesn't seem you can control them with /effort and I always find the output to be better with agents off.

That was actually an interesting case of things that CI/CD don't tend to catch.

It failed to start because it failed to parse the published release notes.

In the CI/CD system it would have passed, because the release notes that broke it, hadn't been published yet.

Those release notes also took down previous versions of claude-code too, rolling back didn't help users.

The breakage wasn't a change in the software, it was a change in the release notes which coincided with the change in the software.

Now, should it have been grabbing release notes and parsing them? No, that's unbelievably dumb (and potentially dangerous), but it wasn't an issue with missing CI/CD, but an interesting case-study in CI/CD gaps and how CI/CD can actually lead to over-confidence.

about once a week I get a claude "auto update" that fails to start with some bun error on our linux machines. It's beyond laughable.

Yeah, git reset --hard is something I do like once a week! lol

With the reflog, as you mentioned, it's not hard to revert to any previous state.

honestly if local LLMs become easier to implement in the future due to dedicated hardware, the Unix-like thing I'm working on might actually get this

200k ought to be enough for anyone.

Yep. If you ask Claude to create a drop-in replacement for an open-source project that passes 100% of the test suite of the project, it will basically plagiarize the project wholesale, even if you changed some of the requirements.

Yeah. I've been enjoying programming with Claude so much I started feeling the need to upgrade to Max. Then it turns out even big companies paying API premiums are getting an intentionally degraded and inferior model. I don't want to pay for Opus if I can't trust what it says.

Ethics don't mean anything when talking about corporations. Their good guy persona is itself a marketing stunt.

https://news.ycombinator.com/item?id=47633396#47635060

> Love your point. Instructions found to be good by trial and error for one LLM may not be good for another LLM.

Well, according to this story, instructions refined by trial and error over months might be good for one LLM on Tuesday, and then be bad for the same LLM on Wednesday.

I would have thought so too. But my n=1 has CC solving pretty much the same task today and about two weeks ago with drastically degraded results.

The background being that we scrapped working on a feature and then started again a sprint later.

In my cynicism I find it more likely that a massively unprofitable LLM company tries to reduce costs at any price than everyone else suffering from a collective delusion.

I agree with you. I too complain about this same phenomenon with my colleagues, and we always arrive at the same conclusion: it’s probably us just expecting more and more over time.

The changes to reduce inference costs are intentional. Last thing you're going to do is have users linger on an older version that spends much more. This is essentially what's going on with layers upon layers of social engineering on top of it.

and all are wrong, but some are more useful than others

True, you can understand an ICE engine all the way down to the chemistry if you so chose. An LLM isn't even understood by its inventors so users have no chance to understand it even if they wanted to.

I’m think a great marketing line for local/selfhosted LLMs in the future - “You can swear at your LLM and nobody will care!”

of course that’s the reason but don’t pretend it’s some user guided decision

I angered the mob elsewhere by being a heretic.

Is that baked in?

No, the receipt had no indication of it being a gift. Was with my family at the time and suddenly started getting $10 extra usage charges every few minutes. I wasn’t able to toggle off the “auto-reload funds” feature until about $180 had been drained from my checkings. For context, here’s the support ticket I sent in on March 7th.

“Hi Anthropic Support,

I'm a Max plan subscriber and I'm writing about approximately $180 in unexpected Extra Usage charges that appeared on my account between March 3-5, 2026. I attempted to resolve this through your Fin AI chatbot (Conversation ID: 215473382652967).

Here's the situation: - I received 16 separate Extra Usage invoices between March 3-5, ranging from $10-$13 each, all charged automatically. - I was not actively using Claude during this period — I was away from my laptop entirely. - When I checked my usage dashboard, it showed my session at 100% usage despite me not using the product. - My API usage dashboard shows only $70 in total lifetime usage, confirming this is not API-related. - My Claude Code session history shows only two tiny sessions from March 5 totaling under 7KB — nowhere near enough activity to generate these charges.

This appears consistent with known billing/usage tracking issues reported by other Max plan users (GitHub issues #29289 and #24727 on the anthropics/claude-code repo), where usage meters show incorrect values and Extra Usage charges accumulate erroneously. However, it is possible that my account was compromised, and I would like assistance determining if that is the case (or if it really is a bug.)

Either way, I am requesting a refund of the Extra Usage charges from March 3-5 only — I do not want to cancel my subscription.”

Still, its on Anthropic to respond to it.

When a third party leaked my CC number which then was used to buy Spotify premium, all it took was 10 minutes of chat with a very polite support agent to have it resolved.

Ignoring the customer is not going to fix it. They'd know if they asked Claude.

Mentioning ULTRATHINK in prompt is the equivalent to /effort max?

I've seen this frequently also

Well they need you to think there is some kind of soul behind it - that is their entire pitch!

Where I live, the law is above some silly terms and conditions.

as someone who uses deepseek, glm and kimi models exclusively, an llm telling me what to do is just off the wall

glm and kimi in particular, they can't stop writing... seriously very eager to please. always finishing with fireworks emoji and saying how pleased it is with the test working.

i have to say to write less documentation and simplify their code.

Haha no that’s change - 4.4x MORE expletives per word in the last week.

Jeez, how fast we get used to alien tech.

You could introduce teleportation boots to humanity and within a few weeks we'd be complaining that sometimes we still have to walk the last 20 meters.

Agreed, the only feedback is switching... however things move fast. Unfortunately that means for me is subscribing or using API for many providers and then just switching models when one gets worse.

If you have a paid plan, you may need to pay for more than one, and "hopefully" the drop in usage (not income) is a good enough signal that there is a issue.

I've actually switched back to the web chat UI and copying Python files for much of my work because CC has been so nerfed.

Maybe a local or intermittent issue? Working for me.

"Evals and vibes" can I put that on a t shirt?

What's that ratio exactly

Are you doing any Digital Twin testing or simulations? I imagine you can't test a product like Claude Code using traditional means.

You cannot control the effort setting sub-agents use and you also cannot use /effort max as a default (outside of using an alias).

No, ultrathink is back and it's the same thing as high effort for the message in which it is included

The note at the end of the post indicates the user asked Claude to review their own chat logs. It's impossible to tell if Claude used or built a a performance harness or just wrote those numbers based on vibes.

There is this 3rd party tracker: https://marginlab.ai/trackers/claude-code/

I've understood that in more recent models you need to run extra compute to get a human-readable version of the thinking tokens, so it does impact latency. Though probably the more important motive is you can squeeze in more concurrent users by skipping this.

So this means I can not resume a session older than 30 days properly?

Generalisations and angry language but I almost agree with the underlying message.

New tools, turbulent methods of execution. There's definitely something here in the way of how coding will be done in future but this is still bleeding edge and many people will get nicked.

Uh, no. Definitely not me at all.

They provide thinking summaries, so I assume they have to call Haiku or some other model to summarise the thinking blocks.

But did they say that about tests they just added themselves too? Had claude try that on me a couple of times >_<

The only regularity I can discern in contemporary online debates about LLMs is that for every viewpoint expressed, with probability one someone else will write in with the diametrically opposite experience.

Today it’s my turn to be that person. Large scientific code base with a bunch of nontrivial, handwritten modules accomplishing distinct, but structurally similar in terms of the underlying computation, tasks. Pointed GPT Pro at it, told it what new functionality I wanted, and it churns away for 40 minutes and completely knocks it out of the park. Estimated time savings of about 3-4 weeks. I’ve done this half a dozen times over the past two months and haven’t noticed any drop off or degradation. If anything it got even better with 5.4.

I’ve had good, alternative experience with my sideproject (adashape.com) where most of the codebase is now written by Claude / Codex.

The codebase itself is architected and documented to be LLM friendly and claude.md gives very strong harnesses how to do things.

As architect Claude is abysmal, but when you give it an existing software pattern it merely needs to extend, it’s so good it still gives me probably something like 5x feature velocity boost.

Plus when doing large refactorings, it forgets much fever things than me.

Inventing new architecture is as hard as ever and it’s not great help there - unless you can point it to some well documented pattern and tell it ”do it like that please”.

i mean you could just search up "is Anthropic making profit" and most sources will say no.

There's this one source on Reddit which calculated that Anthropic has been subsidizing their costs by 32x

you should limit your output (manual or assisted) to a level that is well under your understanding ceiling.

Kernighan’s Law states that debugging is twice as hard as writing. how do you ever intend on debugging something you can’t even write?

do they reason? Where was a video by AI researcher, that showed, that they do not reason but actually come with the result first and then try to invent "reasoning" to match it.

Imagine if all service providers were behaving like this.

> Ahh, sorry we broke your workflow.

> We found that `log_level=error` was a sweet spot for most users.

> To make it work as you expect it so, run `./bin/unpoop` it will set log_level=warn

Yes, LLMs have a very aggressive regression towards the mean - that's probably an existential quality of them.

They are after all, pattern matching.

A lot of humans have difficulty with very reality that they are in fact biological machines, and most of what we do is the same thing.

The funny thing is although I think are are 'metaphysically special' in our expression, we are also 'mostly just a bag of neurons'.

It's not 'natural' for AI to be creative but if you want it to be, it's relatively easy for it to explore things if you prod it to.

I would laugh so hard at this, if your attempt at comparison was not so tragic. Bash and other shells are deterministic. Want to set it just for one user ? - use ~/.bashrc . Set it for all users on the system? use /etc/profile.d/ . Want it just temporary for this session? You got it, environment variables. And it is going to work like that every single time. It is deterministic you see.

Yeah, but for ash/shells these files have wildly different purposes. I don't think it's so distinct with cc.

This should be part of the system prompt. It's absolutely unacceptable to just to not at least try to investigate failures like this. I absolutely hate when it reaches this conclusion on its own and just continues on as if it's doing valid work.

> As I noted in the comment,

Piece of free PR advice: this is fine in a nerd fight, but don't do this in comments that represent a company. Just repeat the relevant information.

> Thinking summaries will now appear in the transcript view (Ctrl+O).

Also: https://github.com/anthropics/claude-code/issues/30958

Wrote my own harness with introspection/long form thinking as a tool that the model can use to plan. Works really well with opus. I can’t use Claude code sadly, it sits there ticking for minutes seemingly doing absolutely nothing although I know it’s working. I hate that as an experience and built my harness with the philosophy of always having something streaming on the ui.

Btw the system prompt length in CC is getting to be insane.

My own experience has been that you really just have to be diligent about clearing your cache between tasks, establishing a protocol for research/planning, and for especially complicated implementations reading line-by-line what the system is thinking and interrupting the moment it seems to be going bad.

If it's really far off the mark, revert back to where you originally sent the prompt and try to steer it more, if it's starting to hesitate you can usually correct it without starting over.

Make sure to use "PRETTY PLEASE" in all caps in your `SOUL.md`. And occasionally remind it that kittens are going to die unless it cooperates. Works wonders.

If OP is correct, Opus has regressed to a point where local models are already on par with it.

Considering the advances in software and hardware, I would expect that in 2 or 3 years.

And I hope we will eventually reach a point where models become "good enough" for certain tasks, and we won't have to replace them every 6 months.

(That would be similar to the evolution of other technologies like personal computers and smartphones.)

We said this since ChatGPT 3. People will never be content with local models.

Ooh, I feel the burn. Care to elaborate? Are you just negging science in general, or ... ?

Fair point on tone. It's a bit of a bind isn't it? When you come with a well-researched issue as OP did, you get this bland corporate nonsense "don't believe your lyin' eyes, we didn't change anything major, you can fix it in settings."

How should you actually communicate in such a way that you are actually heard when this is the default wall you hit?

The author is in this thread saying every suggested setting is already maxed. The response is "try these settings." What's the productive version of pointing out that the answer doesn't address the evidence? Genuine question. I linked my repo because it's the most concrete example I have.

Is somebody saying "you're holding it wrong" a "people willing to help"?

The stated policy of HN is "don't be mean to the openclaw people", let's see if it generalizes.

I mean yeah ultimately it's a tool and I've even leaned off of AI recently for coding because it was exhausting dealing with all its hallucinations

They are definitely that. Regardless of their approach, being upfront and transparent would have been nice. Bricking their own software that previously worked well for their customers isn't cool.

I’m going in circles. Let me take a step back and try something completely different. The answer is a clean refactor.

Wait, the simplest fix is the same hack I tried 45 minutes ago but in a different context. Let me just try that.

Wait,

I think over-thinking is only solved by thinking more, not less. This is only viable once some intelligence threshold is reached, which I think Anthropic has borderline achieved.

  > I think over-thinking is only solved by thinking more, not less.

The cost far outweighs the profits.

Are you using Pi with a cloud subscription, or are you using the API?

Out of curiosity, what can parallel tool calls do that one can't do with parallel subagents and background processes?

you find that pay-per-use API's degraded too?

They run on an operating system you probably don't know all the inner workings of.

And that runs on a chip with trillions of transistors.

Whatever prompting OpenAI has with Codex / GPT 5.4 seems superior here then.

It's very surgical and careful around incremental refactoring, etc. but it also doesn't avoid responsibility.

The problem is degradation. It was working much better before. There are many people (some example of a well know person[0]), including my circle of friends and me who were working on projects around the Opus 4.6 rollout time and suddenly our workflows started to degrade like crazy. If I did not have many quality gates between an LLM session and production I would have faced certain data loss and production outages just like some famous company did. The fun part is that the same workflow that was reliably going through the quality gates before suddenly failed with something trivial. I cannot pinpoint what exactly Claude changed but the degradation is there for sure. We are currently evaling alternatives to have an escape hatch (Kimi, Chatgpt, Qwen are so far the best candidates and Nemotron). The only issue with alternatives was (before the Claude leak) how well the agentic coding tool integrates with the model and the tool use, and there are several improvements happening already, like [1]. I am hoping the gap narrows and we can move off permanently. No more hoops, you are right, I should not have attempted to delete the production database moments.

https://x.com/theo/status/2041111862113444221

https://x.com/_can1357/status/2021828033640911196

Same as how I expect a coin to come up heads 50% of the time.

> how you expect a stochastic model [...] is supposed to be predictable in its behavior.

I used it often enough to know that it will nail tasks I deem simple enough almost certainly.

Imagine a team of human engineers. One day they are 10x ninjas and the next they are blub-coders. Not happening.

Put Claude on PIP.

From this reply, it seems that it has nothing to do with `/effort`: https://github.com/anthropics/claude-code/issues/42796#issue...

I hope you take this seriously. I'm considering moving my company off of Claude Code immediately.

Closing the GH issue without first engaging with the OP is just a slap in the face, especially given how much hard work they've done on your behalf.

I commented on the GH issue, but Ive had effort set to 'high' for however long its been available and had a marked decline since... checks notes... about 23 March according to slack messages I sent to the team to see if I was alone (I wasnt).

EDIT: actually the first glaring issue I remember was on 20 March where it hallucinated a full sha from a short sha while updating my github actions version pinning. That follows a pattern of it making really egregious assumptions about things without first validating or checking. Ive also had it answer with hallucinated information instead of looking online first (to a higher degree than Ive been used to after using these models daily for the past ~6 months)

Gotcha. It seemed though from the replies on the github ticket that at least some of the problem was unrelated to effort settings.

Wait, I should check if they pre-exist on master.

    < 1,000 prompts for compound cd && git commands that can't be safely auto-accepted >

Black boxes aren't inherently bad. But if they don't have well defined mappings of inputs to outputs, they aren't good black boxes. That's the problem with Claude Code imo.

Yet they never give you replies like: oh, you see how dolphins run in the water taking advantage of sea currents if you are talking about boats and speed.

What they will do is to find all the solutions someone did and mix and match around in a mdiocre way of approaching the problem in a much more similar way to a search engine with mix and match than thinking out of the box or specifically for your situation (something also difficult to do anyway bc there will always be some detail missing in the cintext and if you really had go to give all that context each time dumping it from your brain then you would not use it as fast anymore) which humans do infinitely better. At least nowadays.

Now you will tell me that the info is there. So you can bias LLMs to think in more (or less) disruptive ways.

Then now your job is to tweak the LLMs until it behaves exactly how you want. But that is nearly impossible for every situation, because what you want is that it behaves in the way you want depending on the context, not a predefined way all the time.

At that time I wonder if it is better to burn all your time tweaking and asking alternative LLMs questions that, anyway, are not guaranteed to be reliable, or just keep learning yourself about the domain instead of just playing tweaking and absorbing real knowledge (and not losing that knowledge and replace it with machines). It is just stupid to burn several hours in making an expert you cannot check if it says real stuff instead of using that time for really learning about the problem itself.

This is a trade-off and I think LLMs are good for stimulating human thinking fast. But not better at thinking or reasoning or any of that. And if yiu just rely on them the only thing you will emd up being professional at is orompting, which a 16 year old untrained person can do almost as well as any of us.

LLMs can look better if you have no idea of the topic you talk about. However, when you go and check maybe the LLM hallucinated 10 or15% of what it said.

So you cannot rely on it nayways. I still use them. But with a lotof care.

Great for scaffolding. Bad at anything that deviates from the average task.

Equivalent of filing a ticket against the slot machine when you lose more often than expected

The coin's PM is spamming you trivial gaslighting corporate slop, most of it barely edited.

I must admit, the fact that the writing was well formatted and structured was an instant turn off. I did find it insightful. I would have been more willing to read it if it was one lower case run on line with typos one would expect from a prepubescent child. I am both joking and being serious at the same time. What a world.

Yes

Alright, I just tested that setting and it doesn't work.

https://i.imgur.com/MYsDSOV.png

I tested because I was porting memories from Claude Code to Codex, so I might as well test. I obviously still have subscription days remaining.

There is another comment in this thread linking a GitHub issue that discusses this. The GitHub issue this whole HN submission is about even says that Anthropic hides thinking blocks.

What I was pointing out in my comment about the PR advice is that someone responding from a corporation to customers should be providing information to help the customer, nothing more.

Customers may want to fight - you seem to be providing an example - but representatives shouldn't take the bait.

That's interesting research, but I think a more important reason that you don't have access to them (not even via the bare Anthropic api) is to prevent distillation of the model by competitors (using the output of Anthropic's model to help train a new model).

That probably matters for some scenarios, but I have yet to find one where thinking tokens didn't hint at the root cause of the failure.

All of my unsupervised worker agents have sidecars that inject messages when thinking tokens match some heuristics. For example, any time opus says "pragmatic", its instant Esc Esc > "Pragmatic fix is always wrong, do the Correct fix", also whenever "pre-existing issue" appears (it's never pre-existing).

I somewhat understand Anthropic's position. However, thinking tokens are useful even if they don't show the internal logic of the LLM. I often realize I left out some instruction or clarification in my prompt while reading through the chain of reasoning. Overall, this makes the results more effective.

It's certainly getting frustrating having to remind it that I want all tests to pass even if it thinks it's not responsible for having broken some of them.

What's the implication of this? That the model already decided on a solution, upon first seeing the problem, and the reasoning is post hoc rationalization?

But reasoning does improve performance on many tasks, and even weirder, the performance improves if reasoning tokens are replaced with placeholder tokens like "..."

I don't understand how LLMs actually work, I guess there's some internal state getting nudged with each cycle?

So the internal state converges on the right solution, even if the output tokens are meaningless placeholders?

I have seen this to be true many times. The CoT being completely different from the actual model output.

Not limited to Claude as well.

so not only are the sycophantic, hallucinatory, but now they're also proven to be schizophrenic.

neato.

Nah it’s an anti distillation move

So like many of the promises from AI companies, reported chain of thought is not actually true (see results below). I suppose this is unsurprising given how they function.

Is chain of thought even added to the context or is it extraneous babble providing a plausible post-hoc justification?

People certainly seem to treat it as it is presented, as a series of logical steps leading to an answer.

‘After checking that the models really did use the hints to aid in their answers, we tested how often they mentioned them in their Chain-of-Thought. The overall answer: not often. On average across all the different hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time, and DeepSeek R1 mentioned it 39% of the time. A substantial majority of answers, then, were unfaithful.‘

  a9284923-141a-434a-bfbb-52de7329861d
  d48d5a68-82cd-4988-b95c-c8c034003cd0
  5c236e02-16ea-42b1-b935-3a6a768e3655
  22e09356-08ce-4b2c-a8fd-596d818b1e8a
  4cb894f7-c3ed-4b8d-86c6-0242200ea333

Amusingly (not really), this is me trying to get sessions to resume to then get feedback ids and it being an absolute chore to get it to give me the commands to resume these conversations but it keeps messing things up: cf764035-0a1d-4c3f-811d-d70e5b1feeef

I just asked Claude to plan out and implement syntactic improvements for my static site generator. I used plan mode with Opus 4.6 max effort. After over half an hour of thinking, it produced a very ad-hoc implementation with needless limitations instead of properly refactoring and rearchitecting things. I had to specifically prompt it in order to get it to do better. This executed at around 3 AM UTC, as far away from peak hours as it gets.

b9cd0319-0cc7-4548-bd8a-3219ede3393a

> You're right to push back. Let me be honest about both questions.

> The @() implementation is ad-hoc

> The current implementation manually emits synthetic tokens — tag, start-attributes, attribute, end-attributes, text, end-interpolation — in sequence.

> This works, but it duplicates what the child lexer already does for #[...], creating two divergent code paths for the same conceptual operation (inline element emission). It also means @() link text can't contain nested inline elements, while #[a(...) text with #[em emphasis]] can.

I just feel like I can't trust it anymore.

I'll have a look. The CoT switch you mentioned will help, I'll take a look at that too, but my suspicion is that this isn't a CoT issue - it's a model preference issue.

Comparing Opus vs. Qwen 27b on similar problems, Opus is sharper and more effective at implementation - but will flat out ignore issues and insist "everything is fine" that Qwen is able to spot and demonstrate solid understanding of. Opus understands the issues perfectly well, it just avoids them.

This correlates with what I've observed about the underlying personalities (and you guys put out a paper the other day that shows you guys are starting to understand it in these terms - functionally modeling feelings in models). On the whole Opus is very stable personality wise and an effective thinker, I want to complement you guys on that, and it definitely contrasts with behaviors I've seen from OpenAI. But when I do see Opus miss things that it should get, it seems to be a combination of avoidant tendencies and too much of a push to "just get it done and move into the next task" from RHLF.

How much of the code/context gets attached in the /bug report?

how does one actually enforce that? I mean especially for code? You can always just clean room it

I'm not arguing the merits of such a ban, I'm simply stating a fact - that thinking transcripts likely won't return until such a ban is in place.

Because even though it's not representative of the actual thought process, chain of thought improves model performance.

And all hidden in the comments of a niche forum, while the actual issue is closed and whitewashed? You got played.

Surely you realize it's AI responding? (not sure if /s)

IIRC for Enterprise, using /feedback or /bug is an exception to the "we promise not to use your data" agreement.

It is the same with human chain of thought, though. Both of them are post-hoc rationalisations justifying "gut feelings" that come from thought processes the human/agent doesn't have introspection into. And yet asking humans or machines to "think out loud" this way does increase the quality of their work.

> It's contextual though, and pragmatic seems different to me than correct.

To me too, that's why I say they are measurements on different dimensions.

To my mind, I can draw a X/Y axis with "Pragmatic" on the Y and "Correctness" on the X, and any point on that chart would have an {X,Y} value, which is {Pragmatic, Correctness}.

If I am reading the original comment correctly, poster's experience of CC is that it is not an X/Y plot, it is a single line plot, with "Pragmatic" on the extreme left and "Correctness" on the extreme right.

Basically, any movement towards pragmatism is a movement away from correctness, while in my model it is possible to move towards Pragmatic while keeping Correctness the same.

It's better PR to close issues and tell users they're holding it wrong, and meanwhile quietly fix the issue in the background. Also possibly safer for legal reasons.

There's a 5 hour difference between the replies, and new data that came in, so the posts aren't really in conflict.

Also it doesn't sound like they know "there's a model issue", so opening it now would be premature. Maybe they just read it wrong, do better to let a few others verify first, then reopen.

And another where claude gets into a long cycle of "wait thats not right.. hold on... actually..." correcting itself in train of thought. It found the answer eventually but wasted a lot of cycles getting there (reporting because this is a regression in my experience vs a couple weeks ago): 28e1a9a2-b88c-4a8d-880f-92db0e46ffe8

depending on how large your codebase is, hopefully not. At this point use something like the IX plugin to ingest codebase and track context, rather than from the LLM itself.

  - naiveTokens = 19.4M — what ix estimates it would have cost to answer your queries without graph intelligence (i.e., dumping full files/directories into context)                                    
  - actualTokens = 4.7M — what ix's targeted, graph-aware responses actually used
  - tokensSaved = 14.7M — the difference

I can see how that works: this is like building a dependency, a habit if you wish. I think the tighter you couple your workflow to these tools the more dependent you will become and the greater the let-down if and when they fail. And they will always fail, it just depends on how long you work with them and how complex the stuff is you are doing, sooner or later you will run into the limitations of the tooling.

One way out of this is to always keep yourself in the loop. Never let the work product of the AI outpace your level of understanding because the moment you let that happen you're like one of those cartoon characters walking on air while gravity hasn't reasserted itself just yet.

> People who depend on this to make a living must be very angry indeed.

Oh cry me a fucking river.

The people depending on this to make a living don't have the moral high ground here.

They jumped onboard so they could replace other people's living, and those other people were angry too.

They didn't care about that. It's hard to care about them when the thing they depend on to make a living got yanked, because that's what they proposed to do to others.

That Gist does explain quite a few flaws Claude has. I wonder if MEMORY.md is sufficient to counteract the prompt without patching.

I adapted these patches into settings for the tweakcc tool.

https://github.com/Piebald-AI/tweakcc

Pushed it to my dotfiles repository:

https://github.com/matheusmoreira/.files/tree/master/~/.twea...

The tweaks can be applied with

  npx tweakcc --apply

Very interesting. I run Claude Code in VS Code, and unfortunately there doesn't seem to be an equivalent to "cli.js", it's all bundled into the "claude.exe" I've found under the VS code extensions folder (confirmed via hex editor that the prompts are in there).

Edit: tried patching with revised strings of equivalent length informed by this gist, now we'll see how it goes!

I didn't know we could change the base system prompt of Claude Code. Just tried, and indeed it works. This changes everything! Thank you for posting this!

Holy sweet LLM, this gist is crazy. Why did they do this to themselves? I am going to try this at home, it might actually fix Claude.

Is there not a setting to change the system prompt itself? I vaguely remember seeing it in the docs.

  --append-system-prompt
  --append-system-prompt-file
  --system-prompt
  --system-prompt-file

This might be more complex than I imagined. It seems Claude Code dynamically customizes the system prompt. They also update the system prompt with every version so outright replacing it will cause us to miss out on updates. Patching is probably the best solution.

https://github.com/Piebald-AI/claude-code-system-prompts

https://github.com/Piebald-AI/tweakcc