AI recursive self-improvement might not come so quickly after all (August 2026)(technologyreview.com) |
AI recursive self-improvement might not come so quickly after all (August 2026)(technologyreview.com) |
The agent required three interventions during the run. First, we needed to modify the scaffold to resolve a bug in the OpenClaw harness that affected Anthropic reasoning models. Second, we gave the agents a 24-hour deadline extension; at the time of the original deadline, the agents had submitted drafts with a completion report indicating that their self-review was a "Weak Reject" and outlining the next steps they would take if given additional time.
I'm fairly sure Fable 5.1 could have designed a better experiment than the authors here, but hey.
I'd trust anyone but them personally
The former being that these models are helping develop and train future models, but they might not veer too far off in architecture (yet).
The latter is a model being able to train/learn on the fly, in real time, permanently (not just in the current conversation/session), or in other words, adjusting/managing its own weights. But, it also seems like it would take an entire paradigm shift in model architecture from what most LLMs are built on, but I could be wrong.
Test-Time Learning: The model updates its own memory weights while running an inference task.
Meanwhile, Navier–Stokes was solved by an internal model significantly more capable than Astra (and therefore more capable than Mythos/Fable).
I’m afraid this sort of experiment is cope. The labs clearly believe RSI is coming soon.
While I appreciate that the article is throwing a web blanket on doomer claims, the actual study doesn't really get into AI self-improvement. That doesn't require writing papers. That just requires autonomously writing a software system that can produce a better AI agent then the one that created it. That said, I have little worry about this being possible as I have seen no evidence of AI agents being able to produce a working software system of that scale.
Most of the gains in model performance from one release to the next are coming from RLVR post training, which has changed a lot over the last couple of years.
The old way was the model generates a response, then a static verifier looks at the response and evaluates it to assign a reward score. The new way is interactive with an agent running in a custom RL task simulation environment, then scored according to how well it completed the assigned task. For a SOTA model there will be many thousands of these simulation environments, each focusing on trying to teach the model/agent a different skill. Post-training also typically uses training curricula to walk the model up though through different levels of task difficulty.
Training has become very complex.
The job of a post-training AI research engineer consists of things like designing environments, designing training curricula, tweaking learning algorithms, running small scale experiments to verify ideas, etc.
When people talk about RSI, it seems they are mostly talking about automating the job of the post-training research engineer - coming up with new ideas, testing them out, building these environments, etc. At the end of the day there is only so much development speed-up to be had since you still need to actually run those experiments and do the post-training, and are bottle-necked by the amount of compute available to do this. The economics of developing/selling LLMs also requires you to balance development compute cost with revenue generated by the model, so even if you had the spare compute available to put into development, you are ultimately bottle-necked by how fast can the resulting model earn back that sunk cost before you can afford to start the next cycle.
It's not all-or-nothing since some aspects of this automating the job of the post-training research engineer are easier than others, and are already being done, while the job as a whole obviously requires full human intelligence.