Changes to GitHub Copilot individual plans(github.blog) |
Changes to GitHub Copilot individual plans(github.blog) |
The per-request model was pretty insane.
The joke is on them, though (maybe) because this also means that there's literally no reason to keep that account active.
Also, I've never ran into the quota limit before (I only use inline suggestions). The limits have definitely shrunk over time.
And you can then cancel it. I have no idea what a premium request is and it's all just too complicated to use.
Copilot (before today) had one of the simplest & cheapest pricing on the market.
I subscribed two months ago, frustrated with Claude Code and their tight session limits.
The Copilot offer was unbeatable 100 dollars for a 12 months plan, if I remember correctly.
It was pretty clear they were losing money, but hey, it's Microsoft and they need customers, so a competitive push on pricing is expected.
Let's see what these limits look like and I'll decide whether to cancel my subscription or not.
Still a terrible move from them.
Pricing per turn/request was/is an idiotic model and I'm glad they are paying for it. It just forces you into a workflow just to work around business model. Heck the best laugh would be to create a plan outside vscode with interactive CC/Codex then copy paste into GH copilot to do a single session burn of few M tokens.
Again ridiculous model.
This points toward a deeper issue though. We’ll probably see more individual offerings dry up over time. That means you’ll have individuals stuck with hand coding while the hyper productive AI assisted coders will all be at large organizations. If that happens, we’ll enter a phase where computing will once more be available exclusively to the elite few.
They're all operating at a loss, enshittification is coming for us all.
I'm a paying customer and I did not receive ANY communication about this. Was using Opus this afternoon and then it disappeared.
Microsoft really can't stop being Microsoft. I don't dispute the need to charge more for those models, but there is a basic decency to do things and as usual the Big Tech fuckery and complete lack of morals makes them do this in a way that generates total mistrust where it could be just annoyance.
I'll see how Sonnet handles the most difficult problems but I'm foresee a subscription cancelation soon.
Opus 4.6 had a 3x multiplier in Pro. Now the new Opus 4.7 model has 7.5x in Pro+, which offers 5x more requests, but costs 4x more than Pro. So now Opus is essentially 2x the price it used to be.
It’s likely that Sonnet 4.7 will be the new 3x model in Pro — https://github.blog/news-insights/company-news/changes-to-gi...
This whole thing is a massive asshole move, and probably illegal in all countries with a minimum set of consumer protections.
I’ve been using Anthropic models exclusively for the last month on a large, realistic codebase, and I can count the number of times I needed to use Opus on one hand. Most of the time, Haiku is fine. About 10% of the time I splurge for Sonnet, and honestly, even some of those are unnecessary.
Folks are complaining because they lost unlimited access to a Ferrari, when a bicycle is fine for 95% of trips.
Haiku is most definitely not fine for the code bases that I work on. Sonnet is probably fine for most daily tasks, but Opus is still needed to find that pesky bug you've been chasing, or to thoroughly review your PR.
> I’ve been using Anthropic models exclusively for the last month on a large, realistic codebase, and I can count the number of times I needed to use Opus on one hand. Most of the time, Haiku is fine. About 10% of the time I splurge for Sonnet, and honestly, even some of those are unnecessary.
You and I couldn't have more different experiences. Opus 4.7 on the max setting still gets lost and chokes on a lot of my tasks.
I switch to Sonnet for simpler tasks like refactoring where I can lay out all of the expectations in detail, but even with Opus 4.7 I can often go through my entire 5-hour credit limit just trying to get it to converge on a reasonable plan. This is in a medium size codebase.
For the people putting together simple web apps using Sonnet with a mix of Haiku might be fine, but we have a long way to go with LLMs before even the SOTA models are trustworthy for complex tasks.
- If you pay for unlimited trips will you choose the Ferrari or the old VW? Both are waiting outside your door, ready to go.
- Providers that let you choose models don't really price much difference between lower class models. On my grandfathered Cursor plan I pay 1x request to use Composer 2 or 2x request to use Opus 4.6. Until the price is more differentiated so people can say "ok yes Opus is smarter, but paying 10x more when Haiku would do the same isn't worth it" it won't happen.
While I agree with the sentiment, I think that might have been initially driven by older models being nerfed and/or newer ones were better at token/$. And there is this notion that those labs don't constraint the model on the first days after its release.
From a business perspective, why would I start thinking about which model to use, when I could cheaply always use the best model?
I mean at some point some people learn...
I was doing Opus for nasty stuff or otherwise at most planning and then using Sonnet to execute.
Buuuuut I'm dealing with a lot of nonstandard use cases and/or sloppy codebases.
Also, at work, Haiku isn't an enabled model.
But also, if I or my employer are paying for premium requests, then they should be served appropriately.
As it stands this announcement smells of "We know our pricing was predatory and here is the rug pull."
My other lesser worry isn't that Opus 4.7 has a 7.5x multi, it's that the multiplier is quoted as an 'introductory' rate.
Why would it be illegal in any country? Did you pay for an year upfront? Even if so they're offering a pro-rated refund according to the linked blog post:
> If you hit unexpected limits or these changes just don’t work for you, you can cancel your Pro or Pro+ subscription and receive a refund for the time remaining on your current subscription by visiting your Billing settings before May 20
Not sure where the expectation that a business should continue serving you at a given price till the end of time no matter what came from.
The multiplier got 2.5x-ed (from 3 to 7.5)
The minimum plan with Opus access is 4x costlier.
That's a 10x total price increase for having access to Opus at all.
But yes, if you account for the 5x more requests, then it's 2x – not relevant though if you're like me and wouldn't usually max out the quota.
Remember when you are renting other peoples computers they can and will change the terms for their benefit. They own it. You dont. You rent it.
I get the impression that the intersection of HN posters and Copilot users is quite small in practice; that Claude Code and Codex suck up all the oxygen in this room. But it seems plausible we’ll see similar “true costs greatly exceed our current subscription pricing” from Anthropic and OpenAI someday soon…
Opus 4.7 is available today for 7.5 credits per prompt.
They have also suspended new signups.
After testing all of the major IDEs/tools that integrate with LLMs over the last four weeks, I was happy to settle on Copilot. I, and others, seem to be a lot confident in that decision. Especially since there seems to be no refund path for people who prepaid for a year.
In my 30+ years online, I've never seen an industry change so much in terms of pricing, service levels, etc, as I have the last two months.
I'm really curious where all of this lands, and if AI coding tools will be something that only a small percentage can genuinely afford at a competitive level.
Warning: baseless speculation/theorizing ahead.
This is the consequence of LLM inference being really expensive to run, and LLM inference companies being really attractive to VCs. The VC silly money means their costs are totally decoupled from revenue for a while, but I guess eventually people look at incomings vs outgoings and start asking questions.
Previous big trends like SaaS apps, NFTs, blockchain etc were similarly attractive to VCs (for a period of time at least for the last two, the first one is still pretty attractive to VCs), but nowhere near as expensive to run so the behaviour of the companies running them wasn't quite the same.
So:
- DO use AIs to build tools for yourself faster. If the AI goes away, the dashboard and scripts you made will still work.
- DO NOT build your business on top of 3rd party AI services with no way of swapping the backend easily. The question isn't whether there's going to be a "rug-pull", but when it happens. It might be sudden like this one or gradual where they just pump up the price like boiling a frog.
I think this is really telling. The cost of AI has really been masked HUGELY to drive adoption. The true cost is likely to be unsustainable for the big complex tasks (agents running for hours+) that companies have been pushing.
I was skeptical, then quietly bullish on AI, but I'm now seeing signs the market is cracking and the availability is going to receded/costs balloon.
From my simple checks - and from Microsoft's own blog - per token pricing isn't going to be realistic for agentic coding either.
I've been using the Pro+ with Opus 4.6 very successfully and being charged 3x rate was mostly acceptable.
But removing Opus 4.6 and replacing with Opus 4.7 with a 7x rate is just insane!
However, I never expected Opus 4.6 to be removed for the cheaper plans. I expected the pricing model to change, but not to lose access to the model. Moving to being token-based makes sense. It makes the cost more closely aligned with user pricing.
It was nice while it lasted. I got Opus 4.5 to do a lot of work from the beach by assigning it to detailed issues. With this news I've cancelled my Pro subscription. That will help a bit with their capacity issues.
Yes I was hearing that a lot the past few weeks. Its pretty clear what happened:
Anthropic demand soared from OpenClaw, but they were already over-sold. Cowork shipped, Hegseth flexed and a lot of people and entire orgs moved from ChatGPT to Claude in the space of a month. They couldn't handle the demand - they quantized their models, dropped effective usage limits and made all kinds of tweaks in Claude Code to reduce token burn.
Lots of customers fled to Codex and they got crunched as well. Some people noticed Copilot was still selling dollars for nickles and mentioned it to other people.
The only question I care about: Is z.ai next? GLM 5.1 is simply where its at right now. It is not the best model, but it is much better than Sonnet at 1/5 the cost.
It's a shame because the VsCode copilot experience is quite good out of the box compared to all of the other harnesses I've used. But with typical lack of transparency, and sudden, harsh changes... What are they thinking?
After the restrictive rate limiting they've already instituted, I'm simply cancelling and continuing by using providers directly.
I just opted for ChatGPT Pro; I'm trying to take advantage of the increased usage limits they are offering, and from what I heard, Claude Pro was also having bad rate limiting for many people.
Looks like I'm ending my subscription, good (likely too good, no way my account was even remotely within profitable range) access to opus-4.6 was the only reason I used this at all.
I had the exact same issues with the latter - randomly stops working, wipes chat history, just generally seems to be totally broken. But the former works totally fine and still lets you select sonnet/opus. My experience was before this recent 4.6 -> 4.7 change though.
It's the sort of messy job that agents excel at. Decisions need to be made on free text data, translations done into multiple languages, ambiguity handled.
I now need to recheck it still works with another model, which involves a lot of manual verification; and potentially move to Claude Code and pay more money I can ill afford right now.
I'm not even clear from the post when this comes in, I'm guessing effective immediately.
This really hammers home for me the point that we should not be renting our tools.
My own dumb fault for trusting them, I will make sure to learn from this.
Now it's going to cost me an upgrade to $39 Github Pro+ to keep using Opus, and even then it's with much higher multipliers. I don't fully understand the extent to which this reflects actual costs for Opus versus Microsoft leveraging network effects to discourage the usage of a competitor.
I didn't really want to wander outside of VSCode just yet because I was happy with VSCode/Copilot/Opus-4.5 and I don't want to spend all my time experimenting when stuff is changing so fast. But I guess my hand has been forced.
For what its worth, i have been paying for Pro+ and i still got locked out of Opus. I only have access to Opus 4.7 at 7.5x
This was my first thought too but apparently you can just use Claude Code within VSC: https://code.claude.com/docs/en/vs-code
It was clear (see the linked post from 70 days ago) that the current offering was unsustainable, but I'm a bit taken aback at how sharp the clawback is.
I was actually hoping they would change it to something that more closely tracks their actual costs so that they wouldn't have to rug-pull this badly. In particular what was really bad about it was that sending prompts to agents while they were working (to give them corrections) cost extra so I stopped doing that (after initially OpenCode didn't cause billing for that, until they became official).
There are no good solutions for them.
If OpenAI is indeed overbuilt they will completely eliminate Claude.
I guess overall probably was a good decision.
But 7.5x as well as quota limits is pretty hard to swallow.
The annoying thing about the quota limits is they make it really awkward to actually fully utilize the 1500 premium requests you are paying for.
Like if you don’t plan working around the daily and weekly quotas you may not actually be able to utilize your full request allocation.
Claude has the same issue. Single session blows through the quota.
That's not how my creative energy works. I have time that I want to solve problems, and I want to solve them. I don't want a cooldown timer applied to solving a problem. Not to mention the anxiety of realizing that while I sleep I could have burned tokens in that time.
I'm incredibly disappointed when I sat down to my hobbyist programming time and realized copilot was suddenly and dramatically changed in a way that is incredibly disheartening.
Meter my token usage DON'T tell me when I can use them! ARGH.
Think GitHub will do that eventually, just like everyone else is. TFA ends with:
The actions we are taking today enable us to provide the best possible experience for existing users while we develop a more sustainable solution.Guess it’s time to rediscover the lost art of programming without an LLM.
From what I've been gathering, this split in success seems to depend a lot on the types of tasks, the domains / programming languages / frameworks used, and style of prompting.
I couldn't get 5.2 to follow instructions for the life of me, even when repeating multiple times to do / not do something. 5.3-codex was an improvement and 5.4 while _usually_ decent still regularly forgets, goes on unnecessary tangents, or otherwise repeatedly stops just to ask for continuation.
Sure, I'm paying 3x more per request, but I'm also doing 5x fewer requests.
Or well, used to. Still bummed about them dropping 4.6.
given the recent changes that kneecapped the plan for students [1], i feel less bad after seeing this. always had monthly limit on premium requests shown in the extension (which i would watch in dread creep up), the daily/weekly "usage limits" part seem ambiguous at best.
using agentic workloads as the basis for this change does not sit quite right with me. if you look at the newly added debug mode, you may notice the token consumption as well as the subagent/tool calls made behind the scenes. my takeaways:
- it consumes way too much tokens for simple tasks (had one use case where the agent burnt 16+ million tokens just to make 50 line change in a monorepo using plan -> agent approach)
- even when you select a model in the dropdown, the subagents/tools can be called with an entirely different model, often the haiku-4.5. gpt-4o is widely used for creating summaries or titles to display for the plan.
- the new reasoning modes have exacerbated the token burning as the agent tends to loop a whole lot. the prompt vs plan token ratio is quite minuscule, and when combined with your own instruction files and skills, it just goes out of the window.
i think they have given a generous model in the past, but by kneecapping the lower tier, it no longer justifies existence. if they want to raise prices, they can raise the floor. or rather put some work in improving their own orchestration system before putting the blame on the users vibing it out.
Given that they've already silently had session + weekly rate limits for the past couple weeks already at least (I've hit them), I wonder if this change is just making them visible to the user, or if it's actually tightening them too.
If it's the former then I can say they're still significantly more generous than claude pro (on the pro+ plan), so this might be okay. If it's the latter, and the new limits are similar to claude pro then copilot is going to be significantly less useful to me.
It can bill to our Azure sub and I don't have to go through the internal bureaucracy of purchasing a new product/service from a new vendor.
On wait, nevermind.
> The value-add that Microsoft brings to Github Copilot is near zero
The value add is the GitHub integration. By far the best.
GH has cloud agents that can be kicked off from VS Code; deeply integrated with GH and very easy to set up. You can apply enterprise policies on model access, MCP white lists, model behavior, etc. from GitHub enterprise and layered down to org and repo (multiple layers of controls for enterprises and teams). It aggregates and collects metrics across the org.
It also has tight integration with Codespaces which is pretty damn amazing. `gh codespace code` and it's an entire standalone full-stack that runs our entire app on a unique URL and GH credentials flow through into the Codespace so everything "just works". Basically full preview environments for the full application at a unique URL conveniently integrated into GH. But also a better alternative to git worktrees. This is a pretty killer runtime environment for agents because you can fully preview and work on multiple streams at once in totally isolated environments.
If you are a solo engineer, none of this is relevant and probably doesn't make sense (except Codespaces, which is pretty sweet in any case), but for orgs using the GH stack is a huge, huge value add because Microsoft is going to have a better understanding of enterprise controls.
If you want to understand the value add of Copilot, I think you need to spend a bit of time digging into the enterprise account featureset in GH, try Codespaces, try Copilot cloud agents. Then it clicks.
It doesn't matter how competent the actual model is, or how long it's able to operate independently, if the harness can't handle it and drops responses. Made me think are they even using their own harness?
At least Anthropic is obviously dogfooding on Claude Code which keeps it mostly functional.
It was great while it lasted.
If you need some random Egyptian government compliance certification for your vendors or whatever, Microsoft probably has that, Anthropic probably doesn't. Microsoft's (as well as Oracle's) entire deal these days is figuring out what customers care about compliance-wise, and structuring their offerings to deliver exactly that. Whether they're selling their own products, or re-selling somebody else who doesn't have that kind of global footprint and clout, is secondary at best.
Now, it may be the right call to immediately give up and shutdown after Opus 4.5, but models and subscriptions are in flux right now, so the right call is not at all obvious to me.
The agentic AI models could be commoditized, some model may excel in one area of SWE, while others are good for another area, local models may be at least good enough for 80%, and cloud usage could fall to 20%, etc. etc.
Staying in the market and providing multi-model and harness options (Claude and Codex usable in Copilot) is good for the market, even if you don't use it.
Over here in the EU, we need to store sensitive data in an EU server. Anthropic only offers US-hosted version of their models, while G-cloud and Azure has EU based servers.
For direct instance: Anthropic's Claude Code, despite being primarily written in Python, didn't even properly support Windows until far too recently (suggesting people use WSL instead) and even now is not a great Windows experience (requiring git bash, illuminating that among other things Claude's models themselves haven't trained enough on PowerShell, and I try to avoid Claude's models when working on PowerShell scripts still, personally).
Meanwhile VS Code works everywhere I want it to, out of the box, and VS Code's GitHub Copilot integration does the same.
Also, your "near zero" value add includes engineers at Microsoft/Github following the "which is the smartest/most practical model" meta-game for you and just silently updating defaults in Copilot for you without needing to make conscious choices. Sure, you can follow that meta yourself by watching HN every day and sampling hundreds or thousands of opinions across dozens to hundreds of stories each day, then play a "Netflix subscription game" of switching subscriptions every X months when the meta-game shifts or you can pay Microsoft to do all that research for you (which true also includes their professional business relationships/contracts with OpenAI and Anthropic, which is as much of a feature as a bug in my opinion because it's also a signal in the opinion war noise of choosing the "smartest model [for you, right now]" and does show up as its own HN stories for meta-debate). At least to me that's much more than a 0% or 1% value add, but maybe that's also because I don't trust either Anthropic or OpenAI directly, I sort of don't trust HN's comments as a strong guide to playing the meta-game, it's not a meta-game I want to play, and I'm happy to pay someone else to play it for me.
Just for context to the insanity, they allow recursive subagents to I believe its 5 levels deep.
You can make a prompt and tell copilot to dig through a code base, have one sub agent per file, and one Recursive subagent per function, to do some complex codebase wide audit. If you use Opus 4.7 to do this it consumes a grand total of 0.5% of a Pro+ plan.
Thats why this paragraph is here:
> it’s now common for a handful of requests to incur costs that exceed the plan price
This is about personal plans. Github Copilot is half the price of any competition I found.
It's just a decent deal for light users.
However, Claude changed their limits so I got to 100% very easily, and when I did hit 100%, I couldn't be given a "window" of snapshoting my work into something for another agent (either future claude window or GHCP agent) to easily pick up mid-work.
I found the lack of visibility into what costs what was very annoying. For $20/month, you get an arbitrary amount of usage that they were changing without notice or alerts or visuals. I didn't renew CC after it expired and just kept with GHCP.
Even with this announcement from GHCP, I haven't run into a limit. I'm considering upgrading to Pro+ if I don't see a limit.
But I stick with Sonnet more or less in both environments. I only used Opus for a couple of planning sessions at the very beginning, but JIT planning is done good enough by the more mid-tier models.
I have always just used the API, but I decided to give copilot a go on the weekend because of the cheap price. And I am seeing weird behavior like I have never seen before... It will somehow fail to use the file editing tool and then spend an absolutely huge amount of time/tokens building a python script to apply the edit in a sub process... And it will spin it's wheels on stuff the API routinely just gets right in one shot.
Example zed issue https://github.com/zed-industries/zed/issues/54219?issue=zed...
> Claude Code to be removed from Pro Tier? > https://news.ycombinator.com/item?id=47855565
That's the only reason.
In many enterprises you'd need to be very lucky to get an approval for any service that doesn't come from MS.
I never understood the low visibility.
Expensive ram is annoying. I don't look forward to expensive ai.
Enterprise might stick around, but individually, I reckon the developers will flock to OpenCode + open weights (Qwen/GLM/Codestral). The problem then is, if the open weight models impress these new adopters, they will shout about it from rooftops (conferences, social media, blogs) in unison, which might result in an exodus. Especially troublesome considering developers are a major market for both frontier labs (Anthropic & OpenAI) & its IPO ambitions.
I just found out via other news sources, and was surprised I hadn't seen it on HN already.
The change applies to existing subscriptions, some paid a year in advance.
> If you hit unexpected limits or these changes just don’t work for you, you can cancel your Pro or Pro+ subscription and you will not be charged for April usage. Please reach out to GitHub support between April 20 and May 20 for a refund.
Speaking as someone where he only 'real' option we have at work is Copilot Plugin, but I also use Copilot Plugin at home....
This is a shitty shitty shitty move.
As a personal user, I can now only use Opus 4.7 at a 7.5x 'Introductory' multiplier if I upgrade to pro+, but at work I can still apparently do Opus 4.6 at a 3x Multiplier on my work 'enterprise' account.
Honestly it strikes me as though someone at Github Copilot took Palantir's manifesto to heart; Screw the individual, consolidate power to companies on every level.
As far as I can tell, the distinctive feature of my workflow is that I'm giving it small, contained single-commit-sized tasks and limited context. For instance: "For all controller `output()` functions under `Controller/Edit/` and `Controller/Report/`, ensure that they check `Auth::userCanManage`." Others seem to be taking bigger swings.
It felt like I constantly have to go back and either fix things or I just didn't like the results. Like the forward momentum/progress on my projects overally wasn't there over time. Even with tho its cheaper it just doesn't feel worth it, to the point I start to feel negative emotions.
I'm actually a bit worried that I've somehow become to feel more negative emotions with agentic coding. Quicker to feel frustrated somehow when things aren't working.
But seeing that they are stopping to get new subscriptions, and rumours/evidence that they plan to increase coefficients of remaining models, it seems they want us to see "the writing on the wall"
Obviously we’re a long way away from being able to rationally evaluate whether the value of X tokens in model Y is better than model Z, let alone better in terms of developer cost, but that’s kind of where we need to get to, otherwise the model providers are selling magic beans rated in ineffable units of magicalness. The only rational behavior in such a world is to gorge yourself.
I have never had the situation you describe, where Opus won’t come up with “a reasonable plan”, but your definition of “reasonable” might be very different than mine, and of course, running through your credit limit is an entirely tangential problem.
I basically never just yolo large code changes, and use my taste and experience to guide the tools along. For this, Haiku is perfectly fine in nearly all circumstances.
Time to consider buying hardware, I guess
Remember that it's not only the cost per token, but also speed. Some tasks are done faster with simpler/less-thinking models, so it might actually make sense to micromanage the model when you have deadlines.
They removed this now without notice but Wayback Machine still has it: https://web.archive.org/web/20260420190656/https://github.bl...
Usage limits are/were higher in Copilot. They also charge per prompt, not per token.
It is basically a token based pricing, but you get alos a limitation of prompts (you can't just randomly ask questions to models, you have to optimize to make them do the most work for e.g. hour(s) without you replying - or ask them to use the question tool).
The Qwen models are cool, but if you're coming from Opus you will be somewhere between mildly to very disappointed depending on the complexity of your work.
The number of intermediaries that some customers, especially governmental agencies, go through to get just an Azure bill can be wild...
(see https://en.wikipedia.org/wiki/Microsoft_licensing_corruption... ) The upside: the EU finally got a prosecutor. And last but not least everybody forgot why the Baby Bells were born.
1. Go through a 1-2 month procurement process where I have to deal with not only the vendor's sales team on who I'm buying from but also probably multiple teams in my BigCorp. Vendor sales team wants to feel relevant and so I'm sitting in at least one meeting where I'm telling them I just want to buy your shit make it as fast as possible. But then the people in my BigCorp likely not only don't understand why the software is necessary, but need to feel relevant and as such will make me fight through bureaucratic hurdles. I have to get compliance involved. Finance involved. If there's a procurement team I have to get them involved. Probably there's a security questionnaire that my bigcorp's security team uses. I have to send that to the vendor's sales people. They have to send it to their security folks. Security folks on their end have to complete it and send it back. I have to send approvals up the chain on my end, after I've successfully convinced some clueless nontechnical user why software XYZ is important and no, the shit half baked thing we already have doesn't work.
OR alternatively:
2. I can go to the AWS marketplace, click a button, and now my AWS bill goes up X thousands of dollars per month and none of the bullshit from 1 is required. Because AWS is already an approved vendor. Everyone except perhaps someone monitoring the AWS bill for large increases is happy and doesn't care (well, maybe the security team does, but hopefully they aren't tattling on you to the procurement people who have nothing to do and want to stick their fingers in the process and we can make that process go quick), and I just need to tell that person that we are doing it.
It's not always the exact narrative I just laid out, but the gist of it is pretty much procurement at every bigcorp.
Yeah, I hear that a lot, but it never comes with proof. Everyone is special.
I’m sure you’d find that Haiku is pretty functional if there were a constraint on your use.
I don't know how anyone could believe that Haiku is useful for most engineering tasks. I often try to have it take on small tasks in the codebase with well defined boundaries to try to conserve my plan limits, but half the time I end up disappointed and feeling like I wasted more time than I should have.
The differences between the models is vast. I'm not even sure how you could conclude that Haiku is usable for most work, unless you have a very different type of workload than what I work on.
Most importantly, define your acceptance criteria. What do you mean by “disappointed” - this word is doing most of the heavy lifting in your anecdote. (i.e. I know plenty of coders who are “disappointed” by any code that they didn’t personally write, and become reflexively snobby about LLM code quality. Not saying that’s you, but I can’t rule it out, either.)
The models are not the same, but Haiku is definitely not useless, and without a lot more detail, I just ignore anecdotal statements with this sort of hyperbole. Just to illustrate the larger point, I find something wrong with nearly everything Haiku writes, but then again, I don’t expect perfection. I’d probably get a “better” end result for most individual runs with the more expensive models, but at vastly higher cost that doesn’t justify the difference.
But I'm not vibecoding, I don't let models do large work or refactorings, this is just for some small boring tasks I don't want to do.
Maybe, just maybe, the tool isn't suitable for all problem spaces.
I’m not saying that. If anything, it really doesn’t matter much what model you use, and it’s only a case of “you’re holding it wrong” in the sense that you have to use your brain to write code, and that if you outsource your thinking to a machine, that’s the fundamental mistake.
In other words, it’s a tool, not a magic wand. So yeah, you do have to understand how to use it, but in a fairly deterministic way, not in a mysterious woo-woo way.
You were the one who made the claim that Haiku is fine most of the time. To any reasonable person, the burden of proof is on you. Maybe you should share some high level details about your codebase, like its stack, size, problem domain, and so on? Maybe they are so generic that Haiku indeed does fine for you.
You give it 3 examples of the change you want, then ask it to do the other 87. You'll end up saving time and “money”.