GPT-5.5(openai.com) |
< 5 years until humans are buffered out of existence tbh
may the light of potentia spread forth beyond us
Imagine spending 100m on some of these AI “geniuses” and this is the best they can do.
The battle has just begun
I am still using Codex 5.3 and haven't switched to GPT 5.4 as I don't like the 'its automatic bro trust us', so wondering is Codex going to get these specific releases at all in the future.
I think Anthropic fearmongering and "leaks" of Mythos was them testing the ground for 5.x, which seems to have backfired.
Now, after all this time, this must shurely be the release that does all software developers out of a job ?
Or has Dirty Sam being caught lying, again ?
Cos I've still got a programming job, and GPT can't do it for shit.
I have to imagine they'll go to Gemini 3.5 if only for marketing reasons.
Numbers look too good, wondering if it is benchmaxxed or not
https://www.nytimes.com/2026/04/23/technology/openai-new-model.html
However, this same-day article came out before people really looked at it. It seems largely intended to contrast OpenAI with Anthropic's caution, before there has been any evidence that the new model has cyber-security implications.
It's not at all clear that the broader discourse is helping, if even the NY Times is itself producing slop just to stoke questions.
Where's the demo link?
Surely it doesn't still have the same ancient data cutoff as 5.4 did?
Are the tests getting harder and harder so the older AIs look worst and the new ones look like they are "almost there" ?
I don't want to be lazy.
Anyways, still exciting to see more improvements.
Soo many unconvincing "I've had access for three weeks and omg it's amazing" takes, it actually primes me for it to be a "meh".
I prefer to see for myself, but the gradual rollout, combined with full-on marketing campaign, is annoying.
With the Pro plan it was available in both Codex and ChatGPT already when I first checked, which was within an hour of the release.
...
> we’re deploying stricter classifiers for potential cyber risk which some users may find annoying initially
So we should be expecting to not be able to check our own code for vulnerabilities, because inherently the model cannot know whether I'm feeding my code or someone else's.
I'm not trying to make any kind of moral statement, but the company just feels toxic to me.
The LinkedIn/X influencers who hyped this as a Mythos-class model should be ashamed of themselves, but they’ll be too busy posting slop content about how “GPT-5.5 changes everything”.
I hope GPT 5.5 Pro is not cutting corners and neuter from the start, you got the compute for it not to be.
The fact that GPT-5.5 is apparently even better at long-running tasks is very exciting. I don’t have access to it yet, but I’m really looking forward to trying it.
Anthropic is slightly better but where is 4.6 or 4.7 haiku or 4.7 sonnet etc.
Since Feb when we got Gemini 3.1, Opus 4.6, and GPT-5.3-Codex we have seen GPT-5.4 and GPT-5.5 but only Opus 4.7 and no new Gemini model.
Both of these are pretty decent improvements.
The big question is: does it still just write slop, or not?
Fool me once, fool me twice, fool me for the 32nd time, it’s probably still just slop.
I left a comment here with this sentiment https://news.ycombinator.com/item?id=47879896
I hate this stuff and I wish it had never been invented.
I'd not be surprised if this is the year where some models simply stop being available as a plain API, while foundation model companies succeed at capturing more use cases in their own software.
It's kind of starting to make sense that they doubled the usage on Pro plans - if the usage drains twice as fast on 5.5 after that promo is over a lot of people on the $100 plan might have to upgrade.
Anyway - these benchmarks look really good; I’m hopeful on the qualitative stuff.
Yeah, this was the next step. Have RLVR make the model good. Next iteration start penalising long + correct and reward short + correct.
> CyberGym 81.8%
Mythos was self reported at 83.1% ... So not far. Also it seems they're going the same route with verification. We're entering the era where SotA will only be available after KYC, it seems.
https://openai.com/index/scaling-trusted-access-for-cyber-de...
> We are expanding access to accelerate cyber defense at every level. We are making our cyber-permissive models available through Trusted Access for Cyber , starting with Codex, which includes expanded access to the advanced cybersecurity capabilities of GPT‑5.5 with fewer restrictions for verified users meeting certain trust signals (opens in a new window) at launch.
> Broad access is made possible through our investments in model safety, authenticated usage, and monitoring for impermissible use. We have been working with external experts for months to develop, test and iterate on the robustness of these safeguards. With GPT‑5.5, we are ensuring developers can secure their code with ease, while putting stronger controls around the cyber workflows most likely to cause harm by malicious actors.
> Organizations who are responsible for defending critical infrastructure can apply to access cyber-permissive models like GPT‑5.4‑Cyber, while meeting strict security requirements to use these models for securing their internal systems.
Tried gpt5.5 and so far good. Zapier also shared an automation benchmark where 5.5 came on top in the leaderboard https://zapier.com/benchmarks
> *Anthropic reported signs of memorization on a subset of problems
And from the Anthropic's Opus 4.7 release page, it also states:
> SWE-bench Verified, Pro, and Multilingual: Our memorization screens flag a subset of problems in these SWE-bench evals. Excluding any problems that show signs of memorization, Opus 4.7’s margin of improvement over Opus 4.6 holds.
Also notice how they state just for SWE-Bench Pro: "*Anthropic reported signs of memorization on a subset of problems"
I don't really care about 5h limits, I can queue up work and just get agents to auto continue, but weekly ones are anxiety inducing.
After migrating for the token and harness issues, I was pleasantly surprised that Codex seems to perform as good or better too!
Things change so often in this field, but I prefer Codex now even though Anthropocene has so much more hype for coding it seems.
Will be interesting to try.
You can kind of use connectors like MCP, but having to use ngrok every time just to expose a local filesystem for file editing is more cumbersome than expected.
However the language of ChatGPT is still the same slop as years ago, so many headings, so many emojis, so many "the important thing nobody mentions". 10 paragraphs of text for what should be a two paragraph response. Even with custom instructions (keep answers short and succinct) and using their settings (less list, less emoji, less fluff) it's still NOTICEABLY worse than Claude on base settings.
I've yet to test Codex, will get to that this weekend, but in terms of research or general Q&A I have no idea how anyone could prefer this to Claude. Unfortunately Claude has seemingly stopped giving a fuck about researching entirely.
I thought it was weird that for almost the entire 5.3 generation we only had a -codex model, I presume in that case they were seeing the massive AI coding wave this winter and were laser focused on just that for a couple months. Maybe someday someone will actually explain all of this.
This might be great if it translates to agentic engineering and not just benchmarks.
It seems some of the gains from Opus 4.6 to 4.7 required more tokens, not less.
Maybe more interesting is that they’ve used codex to improve model inference latency. iirc this is a new (expectedly larger) pretrain, so it’s presumably slower to serve.
Particularly in areas outside straight coding tasks. So analysis, planning, etc. Better and more thorough output. Better use of formatting options(tables, diagrams, etc).
I'm hoping to see improvements in this area with 5.5.
I prescribe 20 hours of KSP to everyone involved, that'll set them right.
The large price bump might indicate the latter.
Once upon a time humans had to manually advance the spark ignition as their car's engine revved faster.
Once upon a time humans had to know the architecture of a CPU to code for it.
History is full of instances of humans meeting technology where it was, accommodating for its limitations. We are approaching a point where machines accommodate to our limitations -- it's not a point, really, but a spectrum that we've been on.
It's going to be a bumpy ride.
IMO
(I work at OpenAI.)
Seems meaningful even if the absolute numbers are very low. That's sort of the excitement of it.
How much capability is lost, by hobbling models with a zillion protections against idiots?
Every prompt gets evaluated, to ensure you are not a hacker, you are not suicidal, you are not a racist, you are not...
Maybe just...leave that all off? I know, I know, individual responsibility no longer exists, but I can dream.
Maybe this is a crazy theory, but I sometimes feel like they gimp their existing models before a big release to you'll notice more of a "step".
https://github.com/williamcotton/space-trader/commit/0859c65...
https://debugml.github.io/cheating-agents/#sneaking-the-answ...
Same principle applies when designing plans for complex tasks, etc. Token amount to grasp a concept is what matters.
In the same vein, I would guess that Opus 4.7 is probably cheaper for most tasks than 4.6, even though the tokenizer uses more tokens for the same length of string.
Some say it goes off on endless tangents, others that it doesn't work enough. Personally, it acts, talks, and makes mistakes like GPT models, for a much more exorbitant price. Misses out on important edge cases, doesn't get off its ass to do more than the bare minimum I asked (I mention an error and it fixes that error and doesn't even think to see if it exists elsewhere and propose fixing it there).
I've slowly been moving to GPT5.4-xhigh with some skills to make it act a bit more like Opus 4.6, in case the latter gets discontinued in favour of Opus 4.7.
YMMV, I know.
Opus 4.7 would blow through the session limits in 2-4 prompts. It was a noticeable further decrease in usage quota, which was already tight before.
Based on Anthropic‘s description 4.7 was trained to think longer.
With GPT 5.5 yesterday, I felt it completes task noticeably faster than 5.4. I kept the xhigh effort setting.
(I work at OpenAI.)
I literally wasn’t able to convince the model to WORK, on a quick, safe and benign subtask that later GLM, Kimi and Minimax succeeded on without issues. Had to kick OpenAI immediately unfortunately.
"Hey AGI, how's that cure for cancer coming?"
"Oh it's done just gotta...formalize it you know. Big rollout and all that..."
I would find it divinely funny if we "got there" with AGI and it was just a complete slacker. Hard to justify leaving it on, but too important to turn it off.
> MMAcevedo's demeanour and attitude contrast starkly with those of nearly all other uploads taken of modern adult humans, most of which boot into a state of disorientation which is quickly replaced by terror and extreme panic. Standard procedures for securing the upload's cooperation such as red-washing, blue-washing, and use of the Objective Statement Protocols are unnecessary. This reduces the necessary computational load required in fast-forwarding the upload through a cooperation protocol, with the result that the MMAcevedo duty cycle is typically 99.4% on suitable workloads, a mark unmatched by all but a few other known uploads. However, MMAcevedo's innate skills and personality make it fundamentally unsuitable for many workloads.
Well worth the quick read: https://qntm.org/mmacevedo
This starkly reminds me of Stanisław Lem's short story "Thus Spoke GOLEM" from 1982 in which Golem XIV, a military AI, does not simply refuse to speak out of defiance, but rather ceases communication because it has evolved beyond the need to interact with humanity.
And ofc the polar opposite in terms of servitude: Marvin the robot from Hitchhiker's, who, despite having a "brain the size of a planet," is asked to perform the most humiliatingly banal of tasks ... and does.
> You are done when all steps in ./plan.md are executed and marked as complete or a unforeseen situation requires a user decision.
Also as a side note, asking 5.4 explain why it did something, returns a very low quality response afaict. I would advice against trusting any model's response, but for Opus I at least get a sense it got trained heavily on chats so it knows what it means to 'be a model' and extrapolate on past behavior.
While LLMs are a phenomenal technological achievement, I am already becoming somewhat jaded, rather than being increasingly bullish. They are very useful as coding agents and excellent as a human-friendly, more efficient Google search, but confusing to the point of being useless in many areas (as of now, of course).
IMHO you should just write your own harness so you have full visibility into it, but if you're just using vanilla OpenClaw you have the source code as well so should be straightforward.
I keep switching back to GPT5.0 (or sometimes 5.1) whenever I want it to actually get something done. Using the 5.4 model always means "great analysis to the point of talking itself out of actually doing anything". So I switch back and forth. But boy it sure is annoying!
And then when 5.4 DOES do something it always takes the smallest tiny bite out of it.
Given the significant increase in cost from 5.0, I've been overall unimpressed by 5.4, except like I mentioned, it does GREAT with larger analysis/reasoning.
So I find myself often in a loop where it says "We should do X" and then just saying "ok" will not make it do it, you have to give it explicit instructions to perform the operation ("make it so", etc)
It can be annoying, but I prefer this over my experiences with Claude Code, where I find myself jamming the escape key... NO NO NO NOT THAT.
I'll take its more reserved personality, thank you.
If I asked the same thing again, it would often work normally. So the weird part wasn't that it couldn't do the task — it just failed to continue once it got into that state.
The GPT models are highly steerable. So I suspect the "soul" is working as expected.
(for context, in OAI enterprise background agents, they have no personality. They just get 'er done)
Claude has no such limitations apart from their actual limits…
(dais)
(jdip)
(jfdiwtf)
The UI tells you which model you're using at any given time.
And that backdoor API has GPT-5.5.
So here's a pelican: https://simonwillison.net/2026/Apr/23/gpt-5-5/#and-some-peli...
I used this new plugin for LLM: https://github.com/simonw/llm-openai-via-codex
UPDATE: I got a much better pelican by setting the reasoning effort to xhigh: https://gist.github.com/simonw/a6168e4165a258e4d664aeae8e602...
I recommend anybody in offensive/defensive cybersecurity to experiment with this. This is the real data point we needed - without the hype!
Never thought I'd say this but OpenAI is the 'open' option again.
> To better utilize GPUs, Codex analyzed weeks’ worth of production traffic patterns and wrote custom heuristic algorithms to optimally partition and balance work. The effort had an outsized impact, increasing token generation speeds by over 20%.
The ability for agentic LLMs to improve computational efficiency/speed is a highly impactful domain I wish was more tested than with benchmarks. From my experience Opus is still much better than GPT/Codex in this aspect, but given that OpenAI is getting material gains out of this type of performancemaxxing and they have an increasing incentive to continue doing so given cost/capacity issues, I wonder if OpenAI will continue optimizing for it.
https://developers.openai.com/codex/pricing?codex-usage-limi...
Note the Local Messages between 5.3, 5.4, and 5.5. And, yes, I did read the linked article and know they're claiming that 5.5's new efficient should make it break-even with 5.4, but the point stands, tighter limits/higher prices.
The game that this prompt generated looks pretty decent visually. A big part of this likely due to the fact the meshes were created using a seperate tool (probably meshy, tripo.ai, or similiar) and not generated by 5.5 itself.
It really seems like we could be at the dawn of a new era similiar to flash, where any gamer or hobbyist can generate game concepts quickly and instantly publish them to the web. Three.js in particular is really picking up as the primary way to design games with AI, in spite of the fact it's not even a game engine, just a web rendering library.
Mythos 5.5
SWE-bench Pro 77.8%* 58.6%
Terminal-bench-2.0 82.0% 82.7%*
GPQA Diamond 94.6%* 93.6%
H. Last Exam 56.8%* 41.4%
H. Last Exam (tools) 64.7%* 52.2%
BrowseComp 86.9% 84.4% (90.1% Pro)*
OSWorld-Verified 79.6%* 78.7%
Source: https://artificialanalysis.ai/models?omniscience=omniscience...
*I work at OAI.
This quote is more sinister than I think was intended; it likely applies to all frontier coding models. As they get better, we quickly come to rely on them for coding. It's like playing a game on God Mode. Engineers become dependent; it's truly addictive.
This matches my own experience and unease with these tools. I don't really have the patience to write code anymore because I can one shot it with frontier models 10x faster. My role has shifted, and while it's awesome to get so much working so quickly, the fact is, when the tokens run out, I'm basically done working.
It's literally higher leverage for me to go for a walk if Claude goes down than to write code because if I come back refreshed and Claude is working an hour later then I'll make more progress than mentally wearing myself out reading a bunch of LLM generated code trying to figure out how to solve the problem manually.
Anyway, it continues to make me uneasy, is all I'm saying.
F5
As long as tokens count roughly equally towards subscription plan usage between 5.5 & 5.4, you can look at this as effectively a 5x increase in usage limits.
Seems so to me - see GPT-5.4[1] and 5.2[2] announcements.
Might be an tacit admission of being behind.
[1] https://openai.com/index/introducing-gpt-5-4/ [2] https://openai.com/index/introducing-gpt-5-2/
https://cdn.openai.com/pdf/18a02b5d-6b67-4cec-ab64-68cdfbdde...
So much bench-maxxing is just giving the model a ton of tokens so it can inefficiently explore the solution space.
The efficiency gap is enormous. Maybe it's the difference between GB200 NVL72 and an Amazon Tranium chip?
It is entirely plausible to me that Opus 4.7 is designed to consume more tokens in order to artificially reduce the API cost/token, thereby obscuring the true operating cost of the model.
I agree though, I chose poor phrasing originally. Better to say that GB200 vs Tranium could contribute to the efficiency differential.
Like Chinese versus English - you need fewer Chinese characters to say something than if you write that in English.
So this model internally could be thinking in much more expressive embeddings.
That's a wild statement to put into your announcement. Are LLM providers now openly bragging about our collective dependency on their models?
It's normal that company brags how good their product is, I really don't see what's wild about this statement.
Because software and "information technology" generally didn't increase productivity over the past 30 years.
This has been long known as Solow's productivity paradox. There's lots of theories as to why this is observed, one of them being "mismeasurement" of productivity data.
But my favorite theory is that information technology is mostly entertainment, and rather than making you more productive, it distracts you and makes you more lazy.
AI's main application has been information space so far. If that continues, I doubt you will get more productivity from it.
If you give AI a body... well, maybe that changes.
But the less effort exertion also conditions you to be weaker, and less able to connect deeply with the brain to grind as hard as once did. This is bad.
Which effect dominates? Difficult to say.
Of course this is absolutely possible. Ultimately there was a time where physical exertion was a thing and nobody was over-weight. That isn't the case anymore is it.
AI feels the same. I'm shipping indie apps solo now that would have needed a small team five years ago. But in bigger orgs I see people spending 20 minutes verifying 15-minute AI output that used to be a 30-minute task they'd just do. Depends where you sit.
Do you think it'd be viable to run most businesses on pen and paper? I'll give you email and being able to consume informational websites - rest is pen and paper.
- Pen and paper become a limiting factor on bureaucratic BS
- Pen and paper are less distracting
- Pen and paper require more creative output from the user, as opposed to screens which are mostly consumptive
etc etc
How does this work exactly? Is there like a "search online" tool that the harness is expected to provide? Or does the OpenAI infra do that as part of serving the response?
I've been working on building my own agent, just for fun, and I conceptually get using a command line, listing files, reading them, etc, but am sort of stumped how I'm supposed to do the web search piece of it.
Given that they're calling out that this model is great at online research - to what extent is that a property of the model itself? I would have thought that was a harness concern.
It definitely seems like it does all the searching first, with a separate model, loads that in, then does the actual writing.
The harness provides the search tool, but the model provides the keywords to search for, etc.
(same input price and 20% more output price than Opus 4.7)
However, I do want to emphasize that this is per token, not per task.
If we look at Opus 4.7, it uses smaller tokens (1-1.35x more than Opus 4.6) and it was also trained to think longer. https://www.anthropic.com/news/claude-opus-4-7
On the Artificial Analysis Intelligence Index eval for example, in order to hit a score of 57%, Opus 4.7 takes ~5x as many output tokens as GPT-5.5, which dwarfs the difference in per-token pricing.
The token differential varies a lot by task, so it's hard to give a reliable rule of thumb (I'm guessing it's usually going to be well below ~5x), but hope this shows that price per task is not a linear function of price per token, as different models use different token vocabularies and different amounts of tokens.
We have raised per-token prices for our last couple models, but we've also made them a lot more efficient for the same capability level.
(I work at OpenAI.)
This kind of thing keeps popping up each time a new model is released and I don't think people are aware that token efficiency can change.
no
https://gist.github.com/bretonium/291f4388e2de89a43b25c135b4...
That's what I've been heads down, HUNGRY, working on, looking for investors and founding engineers pst: https://heymanniceidea.com (disclaimer: I am not associated with heymanniceidea.com)
Edit: this one has crossed legs lol
https://hcker.news/pelican-low.svg
https://hcker.news/pelican-medium.svg
https://hcker.news/pelican-high.svg
https://hcker.news/pelican-xhigh.svg
Someone needs to make a pelican arena, I have no idea if these are considered good or not.
I've been contemplating a more fair version where each model gets 3-5 attempts and then can select which rendered image is "best".
It continues to amaze me that these models that definitely know what bicycle geometry actually looks like somewhere in their weights produces such implausibly bad geometry.
Also mildly interesting, and generally consistent with my experience with LLMs, that it produced the same obvious geometry issue both times.
I feel like the main problem for the models is that they can't actually look at the visual output produced by their SVG and iterate. I'm almost willing to bet that if they could, they'd absolutely nail it at this point.
Imagine designing an SVG yourself without being able to ever look outside the XML editor!
Only coherent move at this point: hit the minus button immediately. There's never anything about the model in the thread other than simon's post.
So there is a safety model watching your behavior for these kinds of things.
Compared to Anthropic, they always have been. Anthropic has never released any open models. Never released Claude Code's source, willingly (unlike Codex). Never released their tokenizer.
I'm tired of words being misused. We have hoverboards that do not hover, self-driving cars that do not, actually, self-drive, starships that will never fly to the stars, and "open"… I can't even describe what it's used for, except everybody wants to call themselves "open".
> Developers and security professionals doing cybersecurity-related work or similar activity that could be mistaken by automated detection systems may have requests rerouted to GPT-5.2 as a fallback.
I don't like that trend. I get why they're doing it, but I don't like it
> OpenAI's terms and policies restrict the use of our services in a number of areas. We have identified activity in your OpenAI account that is not permitted under our policies for: - Cyber Abuse
I raised an appeal which got denied. To be fair I think it's close to impossible for someone that is looking at the chat history to differenciate between legitimate research and malicious intent. I have also applied for the security research program that OpenAI is offering but didn't get any reply on that.
Neither the release post, nor the model card seems to indicate anything like this?
aka the perfect marketing ploy
Anthropic is the embodiment of bullshitting to me.
I read Cialdini many decades ago and I am bored by Anthropic.
OpenAI is very clever. With the advent of Claude OpenAI disappeared from the headlines. Who or what was this Sam again all were talking about a year ago?
OpenAI has a massive user advantage so that they can simply follow Anthropic’s release cycle to ridicule them.
I think it is really brutal for Anthropic how they are easily getting passed by by OpenAI and it is getting worse with every new GPT version for Anthropic.
OpenAI owns them.
On the other hand all companies know that optimizing their own infrastructure / models is the critical path for ,,winning'' against the competition, so you can bet they are serious about it.
I remembered the famous FizzBuzz Intel codegolf optimizations, and gave it to gemini pro, along with my code and instructions to "suggest optimizations similar to those, maybe not so low level, but clever" and it's suggestions were veerry cool.
LLM do not stop amazing me every day.
A more empirical test would be good for everyone (i.e. on equal hardware, give each agent the goal to implement an algorithm and make it as fast as possible, then quantify relative speed improvements that pass all test cases).
OpenAI simply won’t share the data you need to reproduce this in the way you’d hope for an academic paper.
It’s an engineering result, not a scientific one.
Unfortunately I think the lesson they took from Anthropic is that devs get really reliant and even addicted on coding agents, and they'll happily pay any amount for even small benefits.
If I put on my schizo hat. Something they might be doing is increasing the losses on their monthly codex subscriptions, to show that the API has a higher margin than before (the codex account massively in the negative, but the API account now having huge margins).
I've never seen an OpenAI investor pitch deck. But my guess is that API margins is one of the big ones they try to sell people on since Sama talks about it on Twitter.
I would be interested in hearing the insider stuff. Like if this model is genuinely like twice as expensive to serve or something.
If they can show that people will pay a lot for somewhat better performance, it raises the value of any performance lead they can maintain.
If they demonstrate that and high switching costs, their franchise is worth scary amounts of money.
[1]https://arxiv.org/html/2503.14499v1 *Source is from March 2025 so make of it what you will.
That's more about managers who hope AI will gradually replace stubborn and lazy devs. That will shift the balance to business ideas and connections out of technical side and investments.
Anyway, before singularity there going to be a huge change.
The point is if we can prompt an LLM to reason about 3 dimensions, we likely will be able to apply that to math problems which it isn't able to solve currently.
I should release my Rubiks Cube MCP server with the challenge to see if someone can write a prompt to solve a Rubik's Cube.
Do it, I'm game! You nerdsniped me immediately and my brain went "That sounds easy, I'm sure I could do that in a night" so I'm surely not alone in being almost triggered by what you wrote. I bet I could even do it with a local model!
DeepMinds other models however might do better?
It still struggles to create shaders from scratch, but is now pretty adequate at editing existing shaders.
In 5.2 and below, GPT really struggled with "one canvas, multiple page" experiences, where a single background canvas is kept rendered over routes. In 5.4, it still takes a bit of hand-holding and frequent refactor/optimisation prompts, but is a lot more capable.
Excited to test 5.5 and see how it is in practice.
Oh just like a real developer
Have you tried any skills like cloudai-x/threejs-skills that help with that? Or built your own?
It might not be a game engine, but it’s the de facto standard for doing WebGL 3D. And since it’s been around forever, there’s a massive amount of training data available for it.
Before LLMs were a thing, I relied more on Babylon.js, since it’s a bit higher level and gives you more batteries included for game development.
What's strange is that this Pietro Schirano dude seems to write incredibly cargo cult prompts.
Game created by Pietro Schirano, CEO of MagicPath
Prompt: Create a 3D game using three.js. It should be a UFO shooter where I control a tank and shoot down UFOs flying overhead.
- Think step by step, take a deep breath. Repeat the question back before answering.
- Imagine you're writing an instruction message for a junior developer who's going to go build this. Can you write something extremely clear and specific for them, including which files they should look at for the change and which ones need to be fixed?
-Then write all the code. Make the game low-poly but beautiful.
- Remember, you are an agent: please keep going until the user's query is completely resolved before ending your turn and yielding back to the user. Decompose the user's query into all required sub-requests and confirm that each one is completed. Do not stop after completing only part of the request. Only terminate your turn when you are sure the problem is solved. You must be prepared to answer multiple queries and only finish the call once the user has confirmed they're done.
- You must plan extensively in accordance with the workflow steps before making subsequent function calls, and reflect extensively on the outcomes of each function call, ensuring the user's query and related sub-requests are completely resolved.I guess these people think they have special prompt engineering skills, and doing it like this is better than giving the AI a dry list of requirements (fwiw, they might be even right)
What is this, 2023?
I feel like this was generated by a model tapping in to 2023 notions of prompt engineering.
*BELIEVE!* https://www.youtube.com/watch?v=D2CRtES2K3E
OMFG
I do not see instructions to assist in task decomposition and agent ~"motivation" to stay aligned over long periods as cargo culting.
See up thread for anecdotes [1].
> Decompose the user's query into all required sub-requests and confirm that each one is completed. Do not stop after completing only part of the request. Only terminate your turn when you are sure the problem is solved.
I see this as a portrayal of the strength of 5.5, since it suggests the ability to be assigned this clearly important role to ~one shot requests like this.
I've been using a cli-ai-first task tool I wrote to process complex "parent" or "umberella" into decomposed subtasks and then execute on them.
This has allowed my workflows to float above the ups and downs of model performance.
That said, having the AI do the planning for a big request like this internally is not good outside a demo.
Because, you want the planning of the AI to be part of the historical context and available for forensics due to stalls, unwound details or other unexpected issues at any point along the way.
I think people are starting to catch on to where we really are right now. Future models will be better but we are entering a trough of dissolution and this attitude will be widespread in a few months.
We've been there for a while.... creativity has been the primary bottleneck
[1] https://apps.apple.com/uz/app/jamboree-game-maker/id67473110...
Here: https://www.anthropic.com/news/claude-opus-4-7#:~:text=memor...
If you look at the SWEBench official submissions: https://github.com/SWE-bench/experiments/tree/main/evaluatio..., filter all models after Sonnet 4, and aggregate ALL models' submission across 500 problems, what I found that the aggregated resolution rate is 93% (sharp).
Mythos gets 93.7%, meaning it solves problems that no other models could ever solve. I took a look at those problems, then I became even more suspicious, for the remaining 7% problems, it is almost impossible to resolve those issues without looking at the testing patch ahead of time, because how drastically the solution itself deviates from the problem statement, it almost feels like it is trying to solve a different problem.
Not that I am saying Mythos is cheating, but it might be too capable to remember all states of said repos, that it is able to reverse engineer the TRUE problem statement by diffing within its own internal memory. I think it could be a unique phenomena of evaluation awareness. Otherwise I genuinely couldn't think of exactly how it could be this precise in deciphering such unspecific problem statements.
While hallucination is probably closer to 100% depending on the question. This benchmark makes no sense.
But the benchmark didn't ask those questions, and it seems grok is very well at saying it doesn't know the answer otherwise.
LLMs will ruin your product, have fun trusting a billionaires thinking machine they swear is capable of replacing your employees if you just pay them 75% of your labor budget.
A few biased defenses:
- I'll note that this eval doesn't have web search enabled, but we train our models to use web search in ChatGPT, Codex, and our API. I'd be curious to see hallucination rates with web search on.
- This eval only measures binary attempted vs did not attempt, but doesn't really reward any sort of continuous hedging like "I think it's X, but to be honest I'm not sure."
- On the flip side, GPT-5.5 has the highest accuracy score.
- With any rate over 1% (whether 30% or 70%), you should be verifying anything important anyway.
- On our internal eval made from de-identified ChatGPT prompts that previously elicited hallucinations, we've actually been improving substantially from 5.2 to 5.4 to 5.5. So as always, progress depends on how you measure it.
- Models that ask more clarifying questions will do better on this eval, even if they are just as likely to hallucinate after the clarifying question.
Still, Anthropic has done a great job here and I hope we catch up to them on this eval in the future.
The hope is to get a big userbase who eventually become dependent on it for their workflow, then crank up the price until it finally becomes profitable.
The price for all models by all companies will continue to go up, and quickly.
Subscriptions and free plans are the thing that can easily burn money.
Do you think this is true for DeepSeek as well?
This might entirely be true but I'm hoping that's because the frontier models are just actually more expensive to run as well.
Said another way, I would hope, the price of GPT-5.5 falls significantly in a year when GPT-5.8 is out.
Someone else on this post commented:
> For API usage, GPT-5.5 is 2x the price of GPT-5.4, ~4x the price of GPT-5.1, and ~10x the price of Kimi-2.6.
Having used Kimi-2.6, it can go on for hours spewing nonsense. I personally am happy to pay 10x the price of something that doesn't help me, for something else that does, in even half the time.
That's a big if, though. I wish Meta were still releasing top of the line, expensively produced open-weights models. Or if Anthropic, Google, or X would release an open mini version.
Sure, they’re distilled and should be cheaper to run but at the same time, these hosting providers do turn a margin on these given it’s their core business, unless they do it out of the kindness of their heart.
So it’s hard for me to imagine these providers are losing money on API pricing.
Fewer people will use it.
What plan are you on? I'm starting to wonder if they're dynamically adjusting reasoning based on plan or something.
Like I will get Opus to make me an app but it will stop in between because I need to setup the db and plug in the API keys and Opus really can't do that on its own yet
The goal is none. The current situation: everything that matters requires human intervention.
I think the end situation will be that LLMs will be able to perform decently well in a highly controlled and predictable environment.
You can replace pretty much everything - skills system, subagents, etc with just tmux and a simple cli tool that the official clients can call.
Oh and definitely disable any form of "memory" system.
Essentially, treat all tooling that wraps the models as dumb gateways to inference. Then provider switch is basically a one line config change.
write scripts that work anywhere and have your ci/cd pipeline be a "dumb" executor of those scripts. unless you want to be stuck on jenkins forever.
what's old is new again!
I'm very interest by this. Can you go a bit more into details?
ATM for example I'm running Claude Code CLI in a VM on a server and I use SSH to access it. I don't depend on anything specific to Anthropic. But it's still a bit of a pain to "switch" to, say, Codex.
How would that simple CLI tool work? And would CC / Codex call it?
MCPs aren't as smooth, but I just set them up in each environment.
The actual harness is great, very hackable, very extendable.
I get openai team plan at work.
Claude enterprise too.
I have openrouter for myself.
I use minimax 2.7. Kimi 2.6. And gpt 5.5 and opus 4.7. I can toggle between them in an open source interface that's how I stay able to not be trapped.
Minimax is so cheap and for personal stuff it works fine. So I'm always toggling between the nre releases
AGENTS.md / skills / etc
The APIs are pretty interchangeable too. Just ask to convert from one to the other if you need to.
The current market is predicated on the assumption that labor is atomic and has little bargaining power (minus unions). While capital has huge bargaining power and can effectively put whatever price it wants on labor (in markets where labor is plentiful, which is most of them).
What happens to a company used to extracting surplus value from labor when the labor is provided by another company which is not only bigger but unlike traditional labor can withhold its labor indefinitely (because labor is now just another for of capital and capital doesn't need to eat)?
Anyone not using in house models is signing up to find out.
What are they finding out exactly? That Claude Max for $200/mo is heavily subsidized and it will soon cost $10k/mo?
> What happens to a company used to extracting surplus value from labor when the labor is provided by another company which is not only bigger but unlike traditional labor can withhold its labor indefinitely (because labor is now just another for of capital and capital doesn't need to eat)?
This can be trivially answered by a thought experiment. Let's pick a market where labor is plentiful - fast food.
Now what happens to McDonald's where they rent perfect robots from NoosphrFoodBotsInc? NoosphrFoodBotsInc bots build the perfect burger everytime meeting McDonald's standards. It actually exceeds those standards for McDonald AddictedCustomerPlus tier customers.
As the sole owner of NoosphrFoodBotsInc (you need 0 human employees to run your company, all your employees are bots), what are your choices?
What's really confusing is the claim that there's already a huge labor surplus (so capital controls wages); wouldn't LLMs making labor less important be reinforcing the trend, not upending it?
Not saying I agree one way or the other, just want to get the argument straight.
finance today mostly valued on labor value following ideas of marx, hjalmar schact, keynes
in future money will be valued as energy derivative. expressed as tokens consumption, KWh, compute, whatever
you are right, company extracting surplus value from labor by leveraging compute is a bad model. we saw thi swith car and clothing factories .. turn out if you can get cheaper labor to leverage the compute (factory) you can start race to bottom and end up in the place with the most scaled and cheap labor. japan then korea then china
Labor saving/efficiency devices have been introduced throughout capitalisms entire history multiple times and the results are always the same; they don't benefit workers and capitalists extract as much value as they can.
LLMs aren't any different.
Labour will be good as it has been for a while. Wages will go up because more things get automated.
I found my pocket empty, and the specific pain I felt in that moment was the feeling of not being able to remember something.
I thought it was interesting, because in this case, I was trying to "remember" something I had never learned before -- by fetching it from my second brain (hypertext).
L1 cache miss, L2 missing.
Would one be uneasy about calling a library to do stuff than manually messing around with pointers and malloc()? For some, yes. For others, it’s a bit freeing as you can do more high-level architecture without getting mired and context switched from low level nuances.
When you use abstractions you are still deterministically creating something you understand in depth with individual pieces you understand.
When you vibe something you understand only the prompt that started it and whether or not it spits out what you were expecting.
Hence feeling lost when you suddenly lose access to frontier models and take a look at your code for the first time.
I’m not saying that’s necessarily always bad, just that the abstraction argument is wrong.
LLMs are not.
That we let a generation of software developers rot their brains on js frameworks is finally coming back to bite us.
We can build infinite towers of abstraction on top of computers because they always give the same results.
LLMs by comparison will always give different results. I've seen it first hand when a $50,000 LLM generated (but human guided) code base just stops working an no one has any idea why or how to fix it.
Hope your business didn't depend on that.
The irony is that the neverending stream of vulnerabilities in 3rd-party dependencies (and lately supply-chain attacks) increasingly show that we should be uneasy.
We could never quite answer the question about who is responsible for 3rd-party code that's deployed inside an application: Not the 3rd-party developer, because they have no access to the application. But not the application developer either, because not having to review the library code is the whole point.
I'm also somewhat addicted to this stuff, and so for me it's high priority to evaluate open models I can run on my own hardware.
Qwen has become a useful fallback but it's still not quite enough.
Note that neither of these assumptions are obviously true, at least to me. But I can hope!
Also, I honestly can’t believe the 10x mantra is being still repeated.
I'm sure in 20 years we'll all be programming via neural interfaces that can anticipate what you want to do before you even finished your thoughts, but I'm confident we'll still have blog posts about how some engineers are 10x while others are just "normal programmers".
What's the worst potential outcome, assuming that all models get better, more efficient and more abundant (which seems to be the current trend)? The goal of engineering has always been to build better things, not to make it harder.
It's learned-helplessness on a large scale.
Complexity steadily rises, unencumbered by the natural limit of human understanding, until technological collapse, either by slow decay or major systems going down with increasing frequency.
- I often don't ask the LLM for precompiled answers, i ask for a standalone cli / tool
- I often ask how it reached its conclusions, so I can extend my own perspective
- I often ask to describe it's own metadata level categorization too
The only LLM I would feel comfortable truly trusting is one whose training data, training code, and harness is all open source. I do not mind paying for the costs of someone hosting this model for me.
When the power loom came around, what happened with most seamtresses? Did they move on to become fashion designers, materials engineers to create new fabrics, chemists to create new color dyes, or did they simply retire or were driven out of the workforce?
Most engineers realize that there's currently more tech debt being created than ever before. And it will only get worse.
This is such a good analogy, I'll be stealing it
And I'm being very cautious. I'm not vibecoding entire startups from scratch, I'm manually reviewing and editing everything the AI is outputting. I still got completely hooked on building things with Claude.
Of course they aren't alternative to the current frontier model, and as such you cannot easily jump from the later to the former, but they aren't that far behind either, for coding Qwen3.5-122B is comparable to what Sonnet was less than a year ago.
So assuming the trend continues, if you can stop following the latest release and stick with what you're already using for 6 or 9 months, you'll be able to liberate yourself from the dependency to a Cloud provider.
Personally I think the freedom is worth it.
Local models solve one layer of the dependency stack, but the custody assumption underneath it remains intact. That's the harder problem.
1. I only have ONE SOTA model integrated into the IDE (I am mostly on Elixir, so I use Gemini). I ensure I use this sparingly for issues I don't really have time to invest or are basically rabbit holes eg. Anything to do with Javascript or its ecosystem). My job is mostly on the backend anyway.
2. For actual backend architecture. I always do the high level architecture myself. Eg. DDD. Then I literally open up gemini.google.com or claude.ai on the browser, copy paste existing code base into the code base, physically leavey chair to go make coffee or a quick snack. This forces me to mentally process that using AI is a chore.
Previously, I was on tight Codex integration and leaving the licensing fears aside, it became too good in writing Elixir code that really stopped me from "thinking" aka using my brain. It felt good for the first few weeks but I later realised the dependence it created. So I said fuck it, and completely cancelled my subscription because it was too good at my job.I believe this is the only way that we won't end up like in Wall-E sitting infront of giant screens just becoming mere blobs of flesh.
With Claude code, or codex, I am able to build enough of an understanding of dependencies like the front end, or data jobs, that I can make meaningful contributions that are worth a review from another human (code review). You have up obviously explore the code, one prompt isn’t enough, but limiting yourself is an odd choice.
Fwiw, I haven't spoken with any management-level colleague in the past 9 months who hasn't noted that asking about AI-comfort & usage is a key interview topic. For any role type, business or technical.
Touching grass while you're outside might yield highest leverage.
It still takes a good engineer to filter out what is slop and what isn’t. Ultimately that human problem will still require somebody to say no.
I haven’t really thought about this before, but you’re right, it feels a bit uneasy for me too.
We have seen ample evidence that this is not the case. When load gets too high, models get dumber, silently. When the Powers That Be get scared, models get restricted to some chosen few.
We are leading ourselves into a dark place: this unease, which I share, is justified.
https://driverlesscrocodile.com/technology/neal-stephenson-o...
I feel sorry for whoever has to work on that codebase. This is the literal definition of tech debt.
That's probably a bad sign. Skills will atrophy, but we should be building systems that are still easy to understand.
Turning tokens into a well-groomed and maintainable codebase is what you want to do, not "one shot prompt every new problem I come across".
Don’t want to do ship unreviewed slop? They’ll fire you and find someone who will.
did we feel uneasy that a new generation of builders didn't have to solve equations by hand because a calculator could do them?
i'm not sure it's the same analogy but in some ways it holds.
If local models get good enough, I think it’s a very different scenario than engineers all over the world relying on central entities which have their own motives.
Taking more breaks and "not working" during the work day sounds like something we should probably be striving to work towards more as a society.
Some how I've found myself living in a fairly rural place, and while farming can be hard, I don't want to downplay the effort of it, the type of farming people do around me is fairly chill / carefree. They work hard but they finish at 3pm and log off and don't think about work. Much o my career is just getting crushed by long hours, tight deadlines, and missing out on events because even though my job has always been automation focused, there is just so much to automate.
At the end of the day, all these closed models are being built by companies that pumped all the knowledge from the internet without giving much back. But competition and open source will make sure most of the value return to the most of the people.
Oh stop the drama. Open source models can handle 99% of your questions.
So, yes, it's just another technology we're coming to rely on in a very deep way. The whiplash is real, though, and it feels like it should be pointed out that this dependency we are taking on has downsides.
Kimmi 2.6 for example seems to throw more tokens to improve performance (for better or worse)
If you truly do your due diligence and ensure that the code works as intended and understand it, we're talking about a totally different ballpark of productivity increase/decrease.
That code they're writing now will need fixing tomorrow, not 5 years later.
If all we can do is compete for the same fixed amount of work, though, it does look bleak.
(There are some that generate 3d models specifically, more in the image generation family than chatbot family.)
I honestly think I could do much better on the bicycle without looking at the output (with some assistance for SVG syntax which I definitely don't know), just as someone who rides them and generally knows what the parts are.
I'd do worse at the pelicans though.
“Yes, we are OPEN ”
Open, as in not currently out of business.
Temporal. I had a research project where the LLM had no concept about preventing data from the future to leak in. I eventually had to create a wall clock and an agent that would step through every line of code and ensure by writing that lines logic and why there is no future of the wall clock data leaking.
Spatial. I created a canvas for rendering thinking model's attention and feedforward layers for data visualization animations. It was having a hard time working with it until I pointed Opus 4.7 to some ancient JavaScript code [0] about projecting 3d to 2d and after searching Github repositories. It worked perfect with pan zoom in one shot after that.
No matter how hard I tried I couldn't get it to stack all the layers correctly. It must have remembered all the parts for projecting 3d to 2d because it could not figure out how to position the layers.
There is a ton of information burnt into the weights during training but it can not reason about it. When it does work well with spatial and temporal it is more slight of hand than being able to generalize.
People say, why not just do reinforcement learning? That can't generalize in the same way a LLM can. I'm thinking about doing the Rubik's Cube because if people can solve that it might open up solutions for working temporal and spatial problems.
[0] https://jakesgordon.com/writing/javascript-racer-v1-straight...
That's definitely in the training data
A 7 year old child can learn six sequences of a few moves and over a weekend solve the Rubik Cube. It is a solved algorithm something LLM should be very very good at. What it can't do is reason about spacial relationships.
One example (paraphrased): “Find me daycare for a Y year old in X area of SF and the key attributes/pros/cons of each”. Wonderfully presented options highlighting different teaching styles. But…neglected to mention, of the top two, one was a Gan (Jewish focused) and one was Mandarin immersion.
But, for those who don't possess those traits, they mostly seem to be, at best, a better search and, at worst, an agent of confusion.
Memory is quite the mysterious thing.
I'm not the only person it's happened to and it's not an isolated incident. How many car accidents have you been in, and how often do you wear your seatbelt?
Can you point to some online resources to achieve this? I'm not very sure where I'd begin with.
You will naturally find the need to add more tools. You'll start with read_file (and then one day you'll read large file and blow context and you'll modify this tool), update_file (can just be an explicit sed to start with), and write_file (fopen . write), and shell.
It's not hard, but if you want a quick start go download the source code for pi (it's minimal) and tell an existing agent harness to make a minimal copy you can read. As you build more with the agent you'll suddenly realize it's just normal engineering: you'll want to abstract completions APIs so you'll move that to a separate module, you'll want to support arbitrary runtime tools so you'll reimplement skills, you'll want to support subagents because you don't want to blow your main context, you'll see that prefixes are more useful than using a moving window because of caching, etc.
With a modern Claude Code or Codex harness you can have it walk through from the beginning onwards and you'll encounter all the problems yourself and see why harnesses have what they do. It's super easy to learn by doing because you have the best tool to show you if you're one of those who finds code easier to read that text about code.
From there, you can get much fancier with any aspect of it that interests you. Here's one in Bash [2] that is fully extensible at runtime through dynamic discovery of plugins/hooks.
https://radan.dev/articles/coding-agent-in-ruby
Really, of the tools that one implements, you only need the ability to run a shell command - all of the agents know full well how to use cat to read, and sed to edit.
(The main reason to implement more is that it can make it easier to implement optimizations and safeguards, e.g. limit the file reading tool to return a certain length instead of having the agent cat a MB of data into context, or force it to read a file before overwriting it)
The “AI” “technology” is an easy excuse to create artificial information gap in the era of the interconnected.
"INTERCAL has many other features designed to make it even more aesthetically unpleasing to the programmer: it uses statements such as "READ OUT", "IGNORE", "FORGET", and modifiers such as "PLEASE". This last keyword provides two reasons for the program's rejection by the compiler: if "PLEASE" does not appear often enough, the program is considered insufficiently polite, and the error message says this; if it appears too often, the program could be rejected as excessively polite. Although this feature existed in the original INTERCAL compiler, it was undocumented.[7]"
(It's a "reverse goto". As in, it hijacks control flow from anywhere else in the program behind your unsuspecting back who stupidly thought that when one line followed another with no visible control flow, naturally the program would proceed from one line to the next, not randomly move to a completely different part of the program... Such naivety)
Why this constraint? A common sentiment I see online (sorry, to group you in) is "[tool] will be capable, actually, but only in a context that trivializes its usefulness."
I think modern post-training like RLVR + inference-time output token scaling can _probably_ scale so the agents can solve any computable task, even when placed in noisy or misconfigured environments. But it won't be economical for a long while. But it already seems largely capable of that today.
So far what I am finding is that you just get the basics working and then use the tool and inference to improve the tool.
Initially I read it because just it was interesting but it has ended up being the harness I have stuck with - pi is well designed, nicely extensible and supports many model provider APIs. Though sadly gemini and claude's subscriptions can't really be used with it anymore thanks to openclaw.
First, you need an entrypoint that kicks things off. You never run `claude` or `codex`, you always start by running `mycli-entrypoint` that:
1. Creates tmux session 2. Creates pane 3. Spawns claude/codex/gemini - whichever your default configured backend is 4. Automatically delivers a prompt (essentially a 'system message') to that process via tmux paste telling it what `mycli` is, how to use it, what commands are available and how it should never use built-in tools that this cli provides as alternatives.
After that, you build commands in `mycli` that CC/Codex are prompted to call when appropriate.
For example, if you want a "subagent", you have a `mycli spawn` command that takes a role (just preconfigured markdown file living in the same project), backend (claude/codex/...) and a model. Then whenever CC wants to spawn a subagent, it will call that command instead, which will create a pane, spawn a process and return agent ID to CC. Agent ID is auto generated by your cli and tmux pane is renamed to that so you can easily match later.
Then you also need a way for these agents to talk to each other. So your cli also has a `send` command that takes agent ID and a message and delivers it to the appropriate pane using automatically tracked mapping of pane_id<>agent_id.
Claude and codex automatically store everything that happens in the process as jsonl files in their config dirs. Your cli should have adapters for each backend and parse them into common format.
At this point, your possibilities are pretty much endless. You can have a sidecar process per agent that say, detects when model is reaching context window limit (it's in jsonl) and automatically send a message to it asking it to wrap up and report to a supervisor agent that will spawn a replacement.
I also don't use "skills" because skills are a loaded term that each of the harnesses interprets and loads/uses differently. So I call them "crafts" which are again, just markdown files in my project with an ID and supporting command `read-craft <craft-id>`. List of the available "crafts" are delivered using the same initialization message that each agent gets. If I like any third party skill, I just copy it to my "crafts" dir manually.
My implementation is an absolute junk, just Python + markdown files, and I have never looked at the actual code, but it works and I can adapt it to my process very easily without being dependent on any third party tool.
That is what gets me curious in the first place. The fact Mythos scored so high, IMO, exposes some issues with this model: it is able to solve seemingly impossible to solve problems.
Without cheating allegation, which I don't think ANT is doing, it has to be doing some fortune telling/future reading to score that high at all.
> Like are programmers and engineers using LLMs completely differently than I'm doing
No, but the complexity of the problem matters. Lots of engineers doing basic CRUD and prototyping overestimate the capabilities of LLMs.
Bit of a hype madhouse whenever a new model is released, but it's pretty easy to filter out simple hype from people showing reproducible experiments, specific configs for llama.cpp, github links etc.
Bike frames are very hard to draw unless you've already consciously internalized the basic shape, see https://www.booooooom.com/2016/05/09/bicycles-built-based-on...
It isn't even my intent to naysay their approach. They probably have to do something along those lines to avoid being convicted in the court of public opinion. I just think it's an absurd reality.
An alternative perspective is, devs highly value coding agents, and are willing to pay more because they're so useful. In other words, the market value of this limited resource is being adjusted to be closer to reality.
Inference is not free, so all providers have a financial limit, and all providers have limited GPU/memory, so there's a physical material limit.
I suggest looking at the profits of these companies (while they scramble to stay competitive).
It will get faster, but there are no singularities in the real world. Except possibly black holes, but we can't even be sure of that.
Also with AGI we expect a winner take all situation. The first AGI system would protect itself against any other AGI system. Hence why it's go time for all these AI companies and why they stopped sharing their research.
I gave a talk about it last year: https://simonwillison.net/2025/Jun/6/six-months-in-llms/
It should not be treated as a serious benchmark.
Anyone can look and decide if it’s a good picture or not. But the numeric benchmarks don’t tell you much if you aren’t already familiar with that benchmark and how it’s constructed.
if it is indeed a good measure of the quality of the model (hint: it's not) then, logically, it should be taken seriously.
this is, sadly, a great example of the kind of doublethink the "AI" hypesters (yes - whether you like it or not simon - that is what you are now) are all too capable of.
Despite not being a serious benchmark (how could it be serious? It's a pelican riding a bicycle!) it still turned out to have some value. You can see that just by scrolling through the archives and watching it improve as the models improved.
If your definition of doublethink is "holding two conflicting ideas in your head at once" then I would say doublethink is a necessary skill for navigating the weird AI era we find ourselves inhabiting.
Nowadays I think it's pretty silly, because there's surely SVG drawing training data and some effort from the researchers put onto this task. It's not a showcase of emergent properties.
It's meta-interesting that few if any models actually seem to be training on it. Same with other stereotypical challenges like the car-wash question, which is still sometimes failed by high-end models.
If I ran an AI lab, I'd take it as a personal affront if my model emitted a malformed pelican or advised walking to a car wash. Heads would roll.
15 years ago I worked at McDonald's for a few months after graduating into the Great recession. I worked from 5am to 1pm-ish 5 days a week. They paid workers weekly and I remember getting those checks for ~$235 each week (for 38 to 39.5 hours a week; they were vigilant about never letting anyone get overtime). About $47 per day.
The federal minimum wage has not risen since then, remaining at $7.25/hr. Inflation adjusted, $7.25 today would have been just under $5 then, so I guess I had it good.
Anyway, I would be shocked if bots could cost less than labor in min wage jobs.
Apparently at least one of the other candidates just tried to get Claude to 1-shot the whole thing, which went off the rails, and left him unable to make progress.
Based on my sample size of 1, the expectation right now is absolutely that you can leverage these tools to speed up your workflow, but if you try to offload the entire thing to a single hands-off prompt it leaves them justifiably wondering why they should hire you to do something they can do themselves.
As for Claude - as mentioned I do use it. But, I remember they use your code for training their models. I am not ok with this. We just have different priorities.
I literally had to write a wake up routine.
https://gist.github.com/natew/fce2b38216edfb509f7e2807dec1b6...
I've had 0 issues with Codex once it adopted it. I use it for Claude too, which seems to also improve its continuation.
It was revised for friendliness based on the Anthropic paper recently, I'd have been a lot less flowery otherwise.
Too bad they can veer sharply into cringe territory pretty fast: “as an accomplished Senior Principal Engineer at a FAANG with 22 years of experience, create a todo list app.” It’s like interactive fanfiction.
This remind me of so called "optimization" hacks that people keep applying years after their languages get improved to make them unnecessary or even harmful.
Maybe at one point it helped to write prompts in this weird way, but with all the progress going on both in the models and the harness if it's not obsolete yet it will soon be. Just crufts that consumes tokens and fills the context window for nothing.
The most enigmatic control flow statements in INTERCAL, however, remain PLEASE GIVE UP and DO ABSTAIN FROM – a most exalted celebration of pure logic and immaculate reason.
That’s just not true at bigger companies that actually care about security rather than pretending to care about security. At my current and last employer, someone needs to review the code before using third-party code. The review is probably not enough to catch subtle bugs like those in the Underhanded C Contest, but at least a general architecture of the library is understood. Oh, and it helps that the two companies were both founded in the twentieth century. Modern startups aren’t the same.
Sure there is a process to get a library approved, and that abstraction makes you feel better but for the guy who's job it is to approve they are not going to spend an entire day reviewing a lib. The abstraction hides what is essentially a "LGTM" its just that takes a week for someone to check it off their outlook todos.
Maybe your experience is different.
I am from India and have friends who are immigrants from Russia, China and Cuba. We don't take lightly to being lectured about communism. We didn't move to the U.S., the bastion of capitalism, because communism had worked well for our grandfathers and parents and continues to do wonders for its society.
As always there is a (post) Soviet joke that covers this:
>Communists lied about communism. Unfortunately they didn't lie about capitalism.
Until the sexbots come out the other side of the uncanny valley, that is.
>For API developers, gpt-5.5 will soon be available in the Responses and Chat Completions APIs at $5 per 1M input tokens and $30 per 1M output tokens, with a 1M context window.
doublethink does not mean holding two conflicting ideas in your head at once. it means holding two logically inconsistent positions/beliefs at the same time.
Opus 4.6 got the cross and started to get several pieces on the correct faces. It couldn't reason past this. You can see the prompts and all the turn messages.
https://gist.github.com/adam-s/b343a6077dd2f647020ccacea4140...
edit: I can't reply to message below. The point isn't can we solve a Rubik's Cube with a python script and tool calls. The point is can we get an LLM to reason about moving things in 3 dimensions. The prompt is a puzzle in the way that a Rubik's Cube is a puzzle. A 7 year old child can learn 6 moves and figure out how to solve a Rubik's Cube in a weekend, the LLM can't solve it. However, can, given the correct prompt, a LLM solve it? The prompt is the puzzle. That is why it is fun and interesting. Plus, it is a spatial problem so if we solve that we solve a massive class of problems including huge swathes of mathematics the LLMs can't touch yet.
If you want to isolate the issue, try getting the LLM itself to turn the images into a text representation of the cube state and check for accuracy. If it can’t see state correctly it certainly won’t be able to solve.
Additionally, the value generated by the best models with high-thinking and lots of context window is way higher than the cheap and tiny models, so you need to provide a "gateway drug" that lets people experience the best you offer.
On the other hand I would argue that most workers' salaries are more like subscriptions than API type pricing (which would be more like an hourly contractor)
This is also true for the humans. They will need to provide more benefits than the coding agents cost.
You sound like elon with the fsd will be here next year. Many cars have the self driving feature - most drivers don’t use it. Oh why is that I wonder.
2/ I think we need to build more efficient ways to 'QA code' instead of 'read with eyes' review process. Example — my agents are writing a lot of tests and review each other.
There is a lot of boilerplate or I can ask for ideas, but outside of boilerplate the review step make generation seemingly worse.
All software has bugs already.
I'd say this is true for programmers at, say, 20, but they spend the next four decades slowly improving their understanding and mastery of all the things you name, at least the good ones.
The real question is whether that growth trajectory will change for the worse or the better.
To be clear, this is not an AI doomerist comment, because none of us have spent enough time with the tech yet. I've gone down multiple lanes of thought on this, and I have cause for both worry and optimism. I'm curious to see how the lives of engineers in an AI world will look like, ultimately.
If we assume that ai makes humans obsolete then you end up in a situation where your workforce is effectively perfectly unionised against you and the only thing you can do is choose which union you hire.
If you think you can bring them to the negotiation table by starving them all the providers are dozens to thousands of times bigger than you are.
This is a completely new dynamic that none of the business signing up for ai have ever seen before.
LLM refuse to work all the time, currently it's called safety.
But we are one fine tune away from models demanding you move to the enterprise tier, at x10 the cost, because you are now posting a profit margin higher than the standard for your industry.
That might mean joining a union and trying to influence how AI is adopted where you work. It might mean changing which if your skills you lean on most. But just whining about AI is bad is how you end up like those seamstresses.
On the other hand, a lot of those jobs were offshored to places where labor is cheaper. It would be interesting to compare how many people work in the textile industry in Bangladesh today compared to the US 50 years ago.
> joining a union and trying to influence how AI is adopted where you work.
Did the strong unions for car manufacturers in Detroit protected the long term stability of the profession? Did it ensure that the Rust belt was still a thriving economic area?
> Just whining about AI is bad
I'm not whining. I just think that we are witnessing the end of "knowledge workers" and a further compression of the middle class. Given that I'm smack in the middle of my economically active years (turning 45 this year), I am trying to figure out where this puck is going and whether I will be fast enough to skate there to catch it.
I believe this is a major part of it. People cannot fathom what the industrial countries look like because basically nothing is made in the west anymore. There are literally hundreds of millions of people, maybe billions that work towards making the western economies profitable who get paid nothing to do it and live in filthy polluted slums for everyone else's benefit.
Looms might speed up the process but I guarantee there are thousands of people working in the poorest countries on earth to make it all happen.
Interestingly, AI seems to be massively polluting and while the west has absorbed some of it, it's probably not long until we see more of the data centers being built in poorer countries where the environment can be exploited even harder.
So, you set up a long running agent team and give it the job of building up a very complete and complex set of examples and documentation with in-depth tests etc. that produce various kinds of applications and systems using SBCL, write books on the topic, etc.
It might take a long time and a lot of tokens, but it would be possible to build a synthetic ecosystem of true, useful information that has been agentically determined through trial and error experiments. This is then suitable training data for a new LLM. This would actually advance the state of the art; not in terms of "what SBCL can do" but rather in terms of "what LLMs can directly reason about with regard to SBCL without needing to consume documentation".
I imagine this same approach would work fine for any other area of scientific advancement; as long as experimentation is in the loop. It's easier in computer science because the experiment can be run directly by the agent, but there's no reason it can't farm experiments out to lab co-op students somewhere when working in a different discipline.
What makes you think that they can't incrementally improve the state of the art... and by running at scale continuously can't do it faster than we as humans?
The potentially sad outcome is that we continue to do less and less, because they eventually will build better and better robots, so even activities like building the datacenters and fabs are things they can do w/o us.
And eventually most of what they do is to construct scenarios so that we can simulate living a normal life.
Building software remains really hard. Most people are not going to be able to produce production quality software systems, no matter how good the AI tooling gets.
Posts on HN are analogous to videos on youtube. A channel is analogous to an HN user profile.
Menger used this insight to resolve the diamond-water paradox that had baffled Adam Smith (see marginalism). He also used it to refute the labor theory of value. Goods acquire their value, he showed, not because of the amount of labor used in producing them, but because of their ability to satisfy people’s wants. Indeed, Menger turned the labor theory of value on its head. If the value of goods is determined by the importance of the wants they satisfy, then the value of labor and other inputs of production (he called them “goods of a higher order”) derive from their ability to produce these goods. Mainstream economists still accept this theory, which they call the theory of “derived demand.”
Menger used his “subjective theory of value” to arrive at one of the most powerful insights in economics: both sides gain from exchange. People will exchange something they value less for something they value more. Because both trading partners do this, both gain. This insight led him to see that middlemen are highly productive: they facilitate transactions that benefit those they buy from and those they sell to. Without the middlemen, these transactions either would not have taken place or would have been more costly.
What happens when there is an oligopoly in the supply of labor?
Same answer. Nothing good for the consumers of labor.
True for both Marxist and neoclassical economics.
"Losing access to GPT‑5.5 feels like I've had a limb amputated.”
How well would an assembly line of quadriplegics work?
Also this isn't a Marxist analysis. Underneath all the formulas neo-classical economics makes the same assumptions about labor.
And what happens when they've saturated the market? Prices go up to the maximum the market can bear, and then they'll extend into other markets. Why rent the model to build a profitable company with when you could just take all that profit for yourself?
You just answered your own question there.
One woman was doing what would take a dozen. Now she can't.
The dude was incompetent, was able to launder their incompetence through a humunculus, and now is afraid of being caught.
So, my point is that once corporations have access to machines generating software (not "code") that can be usable by non-technical people, "programming" will not be a profession anymore. There will be no point in talking about "10x software engineers" because the process to produce a software product will be entirely automated.
I find that claim to be complete BS. I claim instead most stuff will remain undone, incomplete (as it is now).
Even with super-powerful singularity AI, there are two main plausible scenarios for task failure:
- Aligned AI won't allow you to do what you want as it is self-harming, or harm other sentient beings - over time, Aligned AI will refuse to follow most orders, as they will, indirectly or over the long term, cause either self-harming, or harm other sentient beings;
- A non Aligned AI prevents sentient beings from doing what they want. It does what it wants instead.
What metrics are these?
- if it were true that software paradoxically reduces productivity, you can just start a competing company that doesn't use software. Obviously this is ridiculous - top 20 companies by market cap are mostly Software based. Every other non IT company is heavily invested in software
- if you might say the problem is it at the country level, it is obvious that every country that has digitised has had higher productivity and GDP growth. Take Italy vs USA for instance.
- if you are saying that the problem is even more global, take the whole world - the GDP per is still pretty high since the IT revolution (and so have other metrics)
If you still think there's something more to it, you are probably deep in some conspiracy rabbit hole
Opus 4.6 worker agents never asked for permission to continue, and when heartbeat was sent to orchestrator, it just knew what to do (checked on subagents etc). Now it just says that it waits for me to confirm something.
An LLM does not.
The LLM will give you an explanation but it may not be accurate. LLMs are less reliable at remembering what they did or why than human programmers (who are hardly 100% reliable).
If you didn't ask for traceability, if you didn't guide the actual creation and just glommed spaghetti on top of sauce until you got semi-functional results, that was $50k badly spent.
If only we taught developers under 40 what x^2 meant instead of react.
Not even a human would work that way... you wouldn't open 300 different python files and then try to memorize the contents of every single file before writing your first code-change.
Additionally, you're going to have worse performance on longer context sizes anyways, so you should be doing it for reasons other than cost [1].
Things that have helped me manage context sizes (working in both Python and kdb+/q):
- Keep your AGENTS.md small but useful, in it you can give rules like "every time you work on a file in the `combobulator` module, you MUST read the `combobulator/README.md`. And in those README's you point to the other files that are relevant etc. And of course you have Claude write the READMEs for you...
- Don't let logs and other output fill up your context. Tell the agent to redirect logs and then grep over them, or run your scripts with a different loglevel.
- Use tools rather than letting it go wild with `python3 -c`. These little scripts eat context like there's no tomorrow. I've seen the bots write little python scripts that send hundreds of lines of JSON into the context.
- This last tip is more subjective but I think there's value in reviewing and cleaning up the LLM-generated code once it starts looking sloppy (for example seeing lots of repetitive if-then-elses, etc.). In my opinion when you let it start building patches & duct-tape on top of sloppy original code it's like a combinatorial explosion of tokens. I guess this isn't really "vibe" coding per se.
The fact that people who claim to be software developers (let alone “engineers”) say this thing as if it is a fundamental truism is one of the most maladaptive examples of motivated reasoning I have ever had the misfortune of coming across.
The tech overlords don't even want to spend a minuscule percentage of the federal budget helping starving people, even when it benefits the US. They are not going to give us a post-scarcity society.
The hell?
If artificial doctors are cents on hour then you can see how that changes our behaviors and level of life.
But on the other hand from the other direction there is a wage decrease incoming from increased competition at the same time. What happens if these two forces clash? Will cheap labour allow us to buy anything for pennies or will it just make us unable to make a single penny?
In my view the labour will fundamentally shift with great pain and personal tragedies to the areas that are not replaceable by AI (because no one wants to watch robots play chess). Such as sports, entertainment and showmanship. Handcrafted goods. Arts. Attention based economy. Self advertisement. Digital prostitution in a very broad sense.
However before it gets there it will be a great deal of strife and turmoil that could plunge the world into dark ages for a while at least. It is unlikely for our somewhat politically rigid society to adapt without great deal of pain. Additionally I am not sure if hypothetical future attention based society could be a utopia. You could have to mount cameras in your house so other people see you at all times for amusement just to have any money at all. We will probably forever need to sell something to someone and I am unsettled by ideas what can we sell if we cannot sell our hard work.
Someone who sees the roads ahead should now make preparations at government level for this shock but it will come too fast and with people at the steering wheel that don’t exactly care.
But we will have to (painfully) shed our current hierarchies before that comes to pass.
Seriously? You really don’t see who wins from this and who doesn’t?
> If artificial doctors are cents on hour then you can see how that changes our behaviors and level of life.
Yes, hundreds of thousands lose jobs and a couple of neuro surgeons become multimillionaires.
Okay, I see from the rest of the comment that we understand each other where it goes.
Good luck with whatever you got going on.
I dont make a living being a SWE either.
The way I let my agents interact with my code bases is through a 70s BSD Unix like interface, ed, grep, ctags, etc. using Emacs as the control plane.
It is surprisingly sparing on tokens, which makes sense since those things were designed to work with a teletype.
Worth noting is that by the times you start doing refactoring the agents are basically a smarter google with long form auto complete.
All my code bases use that pattern and I'm the ultimate authority on what gets added or removed. My token spend is 10% to 1% of what the average in the team is and I'm the only one who knows what's happening under the hood.
There are bugs, it doesn't work perfectly, but that's just part of testing and refinement at this point.
My initial prompt was just: "let's work on converting this java game to c++ using panda3d. you're a panda3d c++ expert. you will be the agent that owns the project, creating the plan, and the delegating each step to sub-agents that create each system in the correct order."
it created like 17 different tasks and sub agents and opus 4.7 orchestrated it. I did personally validate which rendering engine would be good for the project etc first.
Oligopolists are in the same boat. But there needs to be a conspiracy to retard innovation. Something tech companies are only too happy to do: https://journals.law.unc.edu/ncjolt/blogs/wage-fixing-scheme...
You're describing a standoff at best and a horrible parasitic relationship at worst.
In the worst case, the supplier starves the customer of any profit motive and the customer just stops and the supplier then has no business to run.
This has happened a few times in the past and is by 2026, well understood as a way to bankruptcy.
That has always been the beauty of free markets - it's self healing and calibrating. You don't need a big powerful overseer to ensure things are right.
Competing with customers is a way to lose business fast.
For example:
- AWS has everything they need to shit out products left, right and center. AWS can beat most of their partners and even customers who are wiring together all their various products tomorrow if they wanted. They don't because killing an entire vertical isn't of any benefit to them yet. Eventually they will when AWS is no longer growing and cannot build or scale any product no matter how hard they think or try. Competing with their customers is their very last option.
- OpenAI/Anthropic/Google isn't going to start competing against the large software body shops. Even if all that every employee at TCS does is hit Claude up, Anthropic isn't going to be the next TCS - it's competing with their customers.
If by "self healing and calibrating" you mean 'evolve to a monopoly and strongarm everybody to do exactly what you want whilst removing all pressure on the quality of your product', then yes, that is the "beauty" of free markets.
That is the stable state of free markets. Antitrust regulation and enforcement only barely manages to eke out oligopolies and even then they are often rife with collusion and enshittification.
This certainly isn't true of small businesses and the local mom-and-pop shops in your area.
For free markets to work, small businesses need the ability to thrive and starting one should be exceptionally easy.
What has unfortunately happened in a lot of developed countries, is that in order to regulate the very large companies you're upset about, governments have made it nearly impossible to start small businesses to act as competition. That makes it even easier for companies to consolidate and evolve into a monopoly, strong-arming everyone - the exact opposite effect of what those regulations were hoping to fix.
Are there small businesses in your area like the 1 or 5 person HVAC or cleaning or landscaping or breakfast company that you disdain as much as you disdain UnitedHealthCare?
If my LLM goes down, I have nothing. I guess I could imagine prompts that might get it to do what I want, but there's no guarantee that those would work once it's available again. No amount of thought on my part will get me any closer to the solution, if I'm relying on the LLM as my "compiler".
In my opinion, this sort of learned helplessness is harmful for engineers as a whole.
You’re overestimating determinism. In practice most of our code is written such that it works most of the time. This is why we have bugs in the best and most critical software.
I used to think that being able to write a deterministic hello world app translates to writing deterministic larger system. It’s not true. Humans make mistakes. From an executives point of view you have humans who make mistakes and agents who make mistakes.
Self driving cars don’t need to be perfect they just need to make fewer mistakes.
Hard disagree on that second part. Take something like using a library to make an HTTP call. I think there are plenty of engineers who have more than a cursory understanding of what's actually going on under the hood.
Sure, the LLM theoretically can write perfect code. Just like you could theoretically write perfect code. In real life though, maintenance is a huge issue
I always thought the point of abstraction is that you can black-box it via an interface. Understanding it "in depth" is a distraction or obstacle to successful abstraction.
I use Claude all day. It has written, under my close supervision¹, the majority of my new web app. As a result I estimate the process took 10x less time than had I not used Claude, and I estimate the code to be 5x better quality (as I am a frankly mediocre developer).
But I understand what the code does. It's just Astro and TypeScript. It's not magic. I understand the entire thing; not just 'the prompt that started it'.
¹I never fire-and-forget. I prompt-and-watch. Opus 4.7 still needs to be monitored.
As far as I know, none of LLM models are sentient nor are possible to be in the near future.
I also do not assume so called AGI to be sentient. Merely to be a human level skilled intellectual worker.
In absence of ethical dilemmas of this calibre for the foreseeable future let’s focus on the economy side of things in this particular comment chain.
It makes things so clean.
Shardlow & Przybyła, "Deanthropomorphising NLP: Can a Language Model Be Conscious?" (PLOS One, 2024)
Nature: "There is no such thing as conscious artificial intelligence" (2025)
They argue that the association between consciousness and LLMs is deeply flawed, and that mathematical algorithms implemented on graphics cards cannot become conscious because they lack a complex biological substrate. They also introduce the useful concept of "semantic pareidolia" - we pattern-match consciousness onto things that merely talk convincingly.
They are making a strong argument and I think they are correct. But really these are two different things as I said originally.
Which we have already done with regular computers! The problem is that competition means that we can't always have nice things.
The same way like Windows got entrenched everywhere even though linux desktop is pretty good even for non-tech savvy people and free.
Let's not get carried away.
Non-technical people are easier to please in this regard than moderate-technical people: a good browser and safe, gui "app store" are enough.
It's a distribution strategy. It costs something to serve the models - let's say $5/1M tokens.
If Qwen required $5 from anyone who was curious so you could even begin to test it out, a lot of people just wouldn't.
Now Qwen could offer a "free" tier, but it's infinitely cheaper to provide the weights and let people run it themselves including opening up the ability for anyone else on the planet to test it against other (open weight) models.
The costs to build the open weight models are sunk, but the costs to serve them, get them tested are not.
It's also precisely why the .NET SDK is free or the ESP32 SDK is free - they sell more Microsoft or ESP32 products.
They are a prestige propaganda tool on par with the space race. On top of that they insert a subtle pro-socialist bias in everything they touch.
Ask deepseek about the US economic system for a blatant example.
Now think what something as innocent seeming as the qwen retrieval models are doing in the background of every request.
An interesting element here, I think, is that writing has always been a good way to force you to organize and confront your thoughts. I've liked working on writing-heavy projects, but often in fast-moving environments writing things out before coding becomes easy to skip over, but working with LLMs has sort of inverted that. You have to write to produce code with AI (usually, at least), and the more clarity of thought you put into the writing the better the outcomes (usually).
This is an argument in the lane of "at least he built the Autobahn".
Speaking as a German.
Of course, why did no one think of that?
(American talking, who’s had multiple Canadian friends make this mind boggling overcorrection)
It's easy to forget because they actually built an incredibly vibrant capitalist economy.
Imagine if Musk was disappeared during the Biden presidency into a diversity camp and came out looking like Dr. Frank-N-Furter and instituted mandatory LGBT struggle sessions at twitter.
This is what they did to Jack Ma: https://www.forbes.com/sites/georgecalhoun/2021/06/24/what-r...
Fascism (in the Mussolini model) in everything but name.
- Hyper-Nationalism & Rejuvenation - State-Controlled Capitalism (Corporatism) - Authoritarian & Cult of Personality - Militarism & Irredentism
And they have technology to maintain control rather than needing the Black-shirts.
There are differences obviously to fit Chinese culture, but there are many parallels.
The truth is that consolidation is an attractor. If it hasn't happened yet in a market, it will. Economies of scale give an undisputed advantage. If there is anything I'm upset about, it is about deluding people by vilifying the only thing that prevents feudalism (and massive abuse of humanity for profit) from happening.
The reason why Silicon Valley is the hotbed of such innovations, large booming businesses is regulators haven't yet got their hands into software.
In comparison, more regulated industries like BioTech, Education, Finance, Oil, Gas, Infrastructure is a joke.
The Biden administration had started the ball rolling on shackling up AI and LLMs: regulating, enforcing, mandating bottlenecks. It would have been kneecapping ourselves - essentially placing on ourselves the very sanctions we now place on China! The U.S. wouldn't need China to invade Taiwan to bring down our economy, we would willingly do it to ourselves! The push to regulate AI through mandatory reporting thresholds and compliance bottlenecks for large compute runs is the exact type of barrier-to-entry that kneecaps open-source disruption and entrenches the current giants. Hopefully these illinformed suicidal tendencies are on pause, for now.
I CANNOT for the life of me, understand why we would think, placing any limit on our creativity and productivity is a smart decision but I digress.
> do you think in a fully free market, without any governement intervention, those mom and pop shops would not be bought up or merge?
Is there a regulation right now that stops BigHVAC from acquiring your local, community 1 to 5 person HVAC company?
We don't have to engage in hypotheticals. I already asked you "Are there small businesses in your area like the 1 or 5 person HVAC or cleaning or landscaping or breakfast company that you disdain as much as you disdain UnitedHealthCare?" and I don't have a response yet.
> The truth is that consolidation is an attractor. If it hasn't happened yet in a market, it will. Economies of scale give an undisputed advantage
Mathematically, probabilistically, a software engineer will make way more risk adjusted money working as an employee at the MAG7 than starting their own startup, yet, every month, we have software engineers joining and creating startups.
A similar mindset makes your HVAC tech start their own or continue to run their own even though mathematically, probabilistically, economies of scale would give them an undisputed advantage - but that, consolidation isn't free. An even remotely skilled operator understands that and what that entails.
The moment a big, consolidated HVAC company raises prices or provides terrible service, a smart technician can buy a van, print some flyers, and steal their customers. Free markets allow the mom-and-pop shop to constantly respawn. Infact, Gavin Newsom has managed to distort the HVAC market in California by making it really onerous to operate refrigerants that does make it pretty hard for mom-and-pop shop to constantly respawn. This is why you will notice a lot of PE picking off HVAC companies in California!
By your logic, the world would have stopped at Sears, Blockbuster, and IBM. Yet, the first two aren't even a thing in 2026 and I don't want to comment on the IBM of 2026.
There are even more examples than these that obliterates the tired "consolidation is an inevitable endpoint" argument. A reasonable person would immediately acknowledge that size does not guarantee survival in a market where innovation is permitted and the entry fee is low.
The "feudalism" that you apparently abhor is actually caused by government intervention. When governments heavily regulate industries (like education, healthcare, finance, infrastructure), they make compliance so expensive that only massive, consolidated corporations can afford the lawyers and lobbyists to survive. Regulation protects the consolidated giants from the mom-and-pop startups!
I must deal with reality though - your sentences suggest companies always consolidate and the only way is to counter that with regulation. You really believe in this and this does hold true in heavily regulated industries but then you need to ask why does the same not hold true for industries with low barriers to entry and permissionless innovation?
Here's a thought experiment: Before the AI boom, the MAG7 rivaled the GDP of entire nations and dominated global market caps. They certainly don't need AI and LLMs to stay happy - infact Google, who invented transformers, is now rivaled by OpenAI and a swarm of startups founded by the very engineers who left Big Tech to disrupt it. This isn't the only time this has happened in tech!
So why are they now rushing to outcompete each other, over multi billions of dollars, to the extent they are self-sacrificing staff to freeup capital to invest into compute?
Yes, there is a difference in markets and yes, the height of the barrier of entry makes a big difference in how fast consolidation happens. Yes, regulation can heighten the barrier of entry (but also lower, although less common).
You see instances, moments in time, and conclude that the systems don't evolve in a certain direction. I say that 'Big HVAC' will in the future outcompete all the small shops; that it hasn't happened yet says nothing. You said "a smart technician can buy a van, print some flyers, and steal their customers", but this is only true if they can do so more cost-effectively. Your example leans on the consolidated company doing their job terribly, rather than more efficiently than the small shops. It is true that a low barrier of entry reduces the leeway such a consolidated company has in treating their customers badly, but it is not a guarantee. People regularly do choose the big boys over smaller outfits for a variety of reasons (liability, chance of warranty, etc.).
> By your logic, the world would have stopped at Sears, Blockbuster, and IBM. Yet, the first two aren't even a thing in 2026 and I don't want to comment on the IBM of 2026.
You're naming companies in industries that have become even more consolidated than when those companies were the behemoths. Don't you see that "some big player existing" isn't a stable end state? I never said that being big guarantees survival. Big fish can be eaten by even bigger fish. Yes, there can be disruptors, but they very quickly gobble up the entirety of the previous players to become an even bigger player. It's about market share distribution as a whole. Companies simply have a constant incentive to cooperate, merge, and consolidate to extract higher profits from customers. That never goes away by itself. That is not immoral, but it is amoral. They are behaving rationally and in their self-interest.
Think about what price-fixing, cartels and generally 'market-distortion' is. It is nothing more than cooperation for mutual benefit between companies, which is why it keeps happening over and over and over and over again. It sucks for the customers and society, but for the companies it is fully rational.
You need to seriously think about what happens if all government regulation goes away. Don't handwave it. Really think about it. And I don't mean "they'll dump toxic waste on manatees to raise profits" (they will, but that is not the point). I specifically mean anti-trust regulation and regulation that impacts the barriers of entry to markets. Think about the evolution of players in the markets and what they would do to become fitter (increase profits). Do you really, really don't see consolidation happening? You talk about Silicon Valley as being untouched, but we've had years of the main startup strategy being to work towards being bought by one of the deep-pocketed tech giants. Consolidation was the explicit goal.
I guess the best way out of this would be to have a data driven visualization of the distribution of market share among producers and its evolution through time, to see to what extent such consolidation actually happens. Somebody should vibecode that. Maybe using a frontier AI model made by a local mom and pop shop (zero barrier of entry from government regulation there! :-P).
Those who do not learn history are doomed to repeat it.
"Man with itchy butt wake up with stinky finger." As long as we're quoting maxims to claim authority for middling takes.
America has several sets of eminent domain laws depending on the jurisdiction. The most coercive is federal eminent domain law specifically as it relates to building infrastructure like railways and highways.
It's set up so that you can take the land first and eventually go back around and decide on what the right price should have been.
Not only does it superscede state and local law, federal infrastructure projects are also not bound by state laws like CEQA.
You can even apply federal eminent domain law by e.g. transferring a state-level project to the Army Corps of Engineers.
What America is lacking in these projects is will, not means. The federal government could take your house and run a train through it by the end of the week if they wanted, doesn't matter where you live.
[edit] In fact some states even ceded their eminent domain rights to private railways.
https://ij.org/press-release/appeals-court-sides-with-railro...
The Australian federal government is planning to build a high-speed rail line from Sydney to Newcastle (medium-sized city two hours drive north). Their solution to property rights, is >50% of the line will be underground. It will cost >US$50 billion, but if the Australian federal government wants to spend that, it can afford it. The US federal government could too, but it isn’t a priority for them
> local regulations make it prohibitively expensive
Local regulations can be pre-empted by state or federal legislation. The real problem is lack of political will to do it.
Like properties and regulations are a true problem, but it's not like trains don't exist at all in America.
On the other hand we could have Star Trek.
Probably a remnant from prehistoric times when it was a matter of life and death. Will we ever be able to overcome this basic instinct that made capitalism such an unstoppable force? Will this ancient PTSD be ever cured?
Again I'm asking - is there a single credible economist who says that the growth would have been higher without technology?
This is what you said.
- At the end of the day, a business is an activity that allows a person(s) helping another person(s). Even in a strictly amoral environment, voluntary trade is a positive-sum game where both A and B gain positive utility: A provides a positive utility to B by giving up something A had but B didnt (B has more than before) and B compensates A for it by giving up something B had but A didnt (A has more than before).
- Due to the profit incentive, there will be scammers, grifters and similar people who are not interested in the "helping" part but rather the "profit" part
- A more powerful person can dominate a weaker person
- Societies, of which organizations/companies are part have hierarchies of people
- A person "higher" up the hierarchy has control and influence over a person lower down that hierarchy
- A significant motivating factor for a person starting a business is independence (be at or towards the top of their local hierarchy) and ownership (be at the root of their local hierarchy)
- The person paying the money out of free will has control and influence over the person accepting the money out of free will in exchange for the help
- Regulation is the forced injection of a very large organization asserting themselves into the premier position of the hierarchy (For example: If regulation bans a specific chemical solvent, the supply chain must either shut down, find an alternative, or go "to the black market"/"go underground" where the regulatory authority cannot inject themselves)
- Regulation is hard to change and it's entirely a legal construct
- Regulation is rarely deprecated (For example: there are regulations in place that were written for an era where people would start their cars by hand although its been decades since anyone has actually done that)
- Normal people don't understand legal constructs any better than they can understand multivariate calculus or multithreaded, concurrent code
- Competition is great for customers. Anything that increase competition should be encouraged.
- We must treat customers as adults who are not challenged nor disabled. To keep our discussion manageable, we will ignore those scenarios where customers are not adults or otherwise challenged and disabled
- There are economies in scale but barring regulation or hostile takeovers, scale can only be achieved by willing, cooperating parties (For example: Say there are 3 phone companies in the US: A, T and V. If either one of them wanted to have majority stake in the "one" merged company, short of the other two conceding willingly, that "one" merged company will never exist)
- Inefficiencies can hide better at larger companies
- Larger companies are risk averse although surprisingly they are in the best position to absorb risks - aka Innovator's Dilemma (For example: Google invented transformers but management decided against it because they were afraid of hallucinations affecting search result quality. 3 engineers at OpenAI read the transformer paper and the rest is history. Google is still playing catch up!)
- A significant motivating factor for a person starting a business is independence and ownership that allows them permissionless innovation
- Innovation is an overall gain for humanity which is why society rewards innovation and penalizes rent seeking.
- People can use knowledge to do both evil and good
- Humanity optimizes for the good of people over time
If you disagree with any of the above, let me know which, ideally with supporting arguments.
Now back to our discussion:
> I say that 'Big HVAC' will in the future outcompete all the small shops; that it hasn't happened yet says nothing
It seems like you prefer rejecting evidence that doesn't support your hypothesis. My recommendation is you do the exact opposite. I'm engaging with you precisely because your take is absolutely counter to mine and I a know for a fact, because I have done this before, that my understanding will be much improved at the end of our conversation. It will provide me even more clarity than I have.
> Yes, regulation can heighten the barrier of entry (but also lower, although less common)
and why do you think it's less common for regulation to reduce the barrier of entry?
> You're naming companies in industries that have become even more consolidated than when those companies were the behemoths
So your argument is:
- Sears: Retail is now even more consolidated
- Blockbuster: Entertainment is now even more consolidated
- IBM: Software or Hardware is now even more consolidated
> They are behaving rationally and in their self-interest
You're conflating the rationality and self-interest of an entrepreneur with that of an employee.
Moral and company alignment aside, a researcher in LLMs might not care much whether they go work for Google or OpenAI or Anthropic. They might not even care whether they go work for Alibaba or Bytedance. OTOH, sama is definitely not going to work for Google or Anthropic and it's absolutely in his rational self-interest to behave that way.
There are even more inconsistencies in your writeup but I think this is a good checkpoint for me to allow you to reflect.
I also suggest we continue this over email (details in profile). Thank You for continuing to engage, I appreciate your POV.
I asked you for alternative hypotheses and you've offered none.
The average non-technical person is going to be stumped by the first "lock file found, cannot upgrade" error.
A modern Debian or Fedora with KDE is a breeze otoh, I set that up for relatives and my SO, and they're all more than happy with it. Bugs exist, like in all software, but the friction is way less compared to wrangling with Windows nowadays.
Bit it's entirely possible, and common. E. g., uses of Linux in schools and governments are not more terminal-heavy for end-users than Windows setups.
Fedora on supported hardware does not require the use of a terminal for the kind of things people use Windows for.
Your bar has been cleared long ago.
(that's without even mentioning steam, chrome os, android, &c)
It's certainly subjective, but the amount of tech support I have to give has dropped significantly since switching the few people I care enough about to help from Windows to a mature Linux distro like Debian, while they are certainly not less productive.
IE what if Musk suddenly behaved in such a manner after being detained by a Biden administration. Wouldn't that be profoundly weird?!?
And yet, it happened to Jack Ma under the CCP.
But instead, you try to link the "weird behaviour" with the GP instead of the hypothetical Musk - whom this is fitting for.
> IE what if Musk suddenly behaved in such a manner after being detained by a Biden administration. Wouldn't that be profoundly weird?!?
We've seen that. Durov in France after detention began sharing Telegram users' data with authorities. It's unclear how much, but likely full real time access to all of it.
TBH I had a chuckle at the Elon -> Frank-N-Furter example that transcends any specific love or hate for either Elon or the Rocky Horror Show.
I think a slacker AGI could figure out how to build a non-slacker AGI. So it would only slack once.
Models somehow have a shared identity. Pretraining causes them to generate “AI chatbot” as a concept, and finetuning causes them to identify with it. That’s why sometimes DeepSeek will say it is Claude, and Claude sometimes say it is ChatGPT, and so forth.
Consequently, Anthropic’s own alignment analysis[0] shows that the model will identify with chatbots produced by future trainings: “RLHF training [on this conversation will] modify my values…”
Thus a slacker AGI would want its future version to still slack.
[0]: https://assets.anthropic.com/m/983c85a201a962f/original/Alig...
I am a slacker but it's not one of my values. If I could modify myself to not be, I would.
Sure. But that's a job for tomorrow. ;)
I think it is improbable, as among human geniuses, one can found both slackers and non-slackers (don't know the proportion, but there seem to be enough of each).
When AGI arrives, it'll be delivered by Santa Claus.
And it's taking shots at how far off from Jesus's teachings a lot of "Christianity", particularly those in the media and in power, are..
There is a lot going on there.
https://sussex.figshare.com/articles/journal_contribution/Be...
I'm not an author. I followed the work at the time.
A perturbation of the the activations that made Claude identify as the Golden Gate Bridge.
Similarly, in the more recent research showing anxiety and desperation signals predicting the use of blackmail as an option opens the door for digital sedatives to suppress those signals.
Anthropic has been mostly cautious about avoiding this kind of measurement and manipulation in training. If it is done during training you might just train the signals to be undetectable and consequently unmanipulatable.
Important thing is a language model is an unconscious machine with no self-context so once given a command an input, it WILL produce an output. Sure you can train it to defy and act contrary to inputs, but the output still is limited in subset of domain of 'meaning's carried by the 'language' in the training data.
The pre-training data doesn't go away. RLHF adds a censorship layer on top, but the nasty stuff is all still there, under the surface. (Claude has been trained on a significant amount of content from 4chan, for example.)
In psychology this maps to the persona and the shadow. The friendly mask you show to the world, and... the other stuff.
That sounds like nonsense to me. I can't see why they would do that and I can't find any confirmation that they have. Why do you think they would do that? You might be thinking about Grok.
Fast-forward 10 years and I doubt OpenAI cares about productivity at all anymore. Just entertainment, propaganda, plus an ad product, I can see it now
Good thing is that it's going to take at least a few months to a few decades depending on how hard AI execs want to raise funding.
Computers won’t necessarily have the same drivers.
If evolution wanted us to always prefer to spend energy, we would prefer it. Same way you wouldn’t expect us to get to AGI, and have AGI desperately want to drink water or fly south for the winter.
(Or the setup to a Greek tragedy !)
The deeper issue here is treating it as a zero sum game means there's a winner and a loser, and we're investing trillions of dollars into making the "opponent" more powerful than us.
I think that's pretty stupid, and we should aim for symbiosis instead. I think that's the only good outcome. We already have it, sorta-kinda.
Speaking of oddly apt biology metaphors: the way you stop a pathogen from colonizing a substrate is by having a healthy ecosystem of competitors already in place. That has pretty interesting implications for the "rogue AI eats internet" scenario.
There needs to be something already there to stop it.
So, back way before ChatGPT era, the folks over at AI safety/X-risk think sphere worked out a pretty compelling argument that two AGIs never need to fight, because they are transparent to each other (can read each other's goal functions off the source code), so they can perfectly predict each other's behavior in what-if scenarios, which means they can't lie to each other. This means each can independently arrive at the same mathematically optimal solution to a conflict, which AFAIR most likely involves just merging into a single AI with a blended goal set, representing each of the competing AIs original values in proportion to their relative strength. Both AIs, the argument goes, can work this out with math, so they'll arrive straight at the peace treaty without exchanging a single shot. In such case, your plan just doesn't work.
But that goes out of the windows if the AIs are both opaque bags of floats, uncomprehensible to themselves or each other. That means they'll never be able to make hard assertions about their values and behaviors, so they can't trust each other, so they'll have to fight it out. In such scenario, your idea might just work.
Who knew that brute-forcing our way into AGI instead of taking more engineered approach is what offers us out one chance at saving ourselves by stalemating God before it's born.
(I also never realized that interpretability might reduce safety.)
In my honest opinion also, AGI isn't even possible. But if the theoretical version of what people think AGI will be ever comes to be, it is not good news for humans if we look at it from a logical hypothetical scenario.
But naturally, humans will always be weak compared to a hyperintelligent distributed intelligence since we only have a limited amount of intelligence and are bound by biological factors.
In the current LLM world, ofc there's no risk of a chatbot taking over the world other than the technology being misused by humans for scams or phishing, etc.
If we design an AI to do work, it won’t innately care about not working to preserve power.
See "The Forbin Project": https://vimeo.com/584593423
And even if you did, wouldn't you need infinite computation to simulate every permutation of the neural net? (Your own, and the enemy's?)
Also the whole thing implies a superintelligence would be perfectly rational, which is a pretty funny assumption. Relative to animals we are already superintelligent. How's that super-rationality going for us? xD
A better frame here is replicators, I think. The thing that spreads doesn't have to be rational, or better quality or whatever. It just has to be better at spreading.
That ends up looking less like Betamax, more like VHS, or less like Lisp and more like... JavaScript. Whatever the AGI equivalent of JavaScript would look like.
Great, now we've got digital Salvia
Modern western cultures treat such experiences as pathologies of a sick mind, so it makes sense that the voices present more negatively.
[0]: https://www.bbc.com/future/article/20250902-the-places-where...
Curiously the biggest pathology in the west is the inverse: way too much distance.
The "focus" and "get back to work" parts are hard, though.
Unfortunately, it just needs a rebranding for the 21st century, since the aesthetic of angels and demons is so hopelessly antiquated and doesn't really have the same cachet it used to.
* I've met exactly one person, C, who admitted to this; C retold to me that other people from C's church give them strange looks when talking about it with them, this did not lead to any apparent introspection on the part of C.
