Even if we had an algorithm to incrementally update weights without catastrophic forgetting (I don't believe we do - but doesn't seem like such a tough problem), then this implies that everyone has their own personalized model, else if you combine all these updates there is no data privacy. Cloud serving also really depends on everyone using the same model so that you can batch requests and not reload weights for each user.

The much more achievable goal, without needing to upend the whole serving model, would be just to implement continual "episodic" memorization (text-only maybe), but even with compaction/consolidation the next question would be how do you retrieve these external memories and get the necessary chunks into context when needed. Some sort of vector store, perhaps? How do you avoid vendor lock-in - perhaps have agents store/retrieve vector-store context chunks from a vendor-agnostic cloud store?

So, even the most basic form of continual-learning-like enhancement becomes tricky. What you'll first see is presumably just enhancements of the vendor-specific memory mechanisms that are already available.

What I would consider as true continual learning, close to what an animal or human does, would require much more extensive architectural and deployment/business model changes - since then we're really talking about more than just an update/recall problem, but rather the whole autonomous agentic loop of predicting/acting/failing/learning/etc with innate traits like curiosity (prediction failure) and boredom to make sure the agent is exposed to learning situations in the first place. At this point you are building an artificial brain, not just an LLM. Some companies such as Google/DeepMind have a more ambitious definition of AGI (more than just an LLM) that is perhaps a step in this direction.

Even with this sort of animal-like continual exploration/learning, you still wouldn't have something that is human level, but at least much closer in terms of ability to learn.

The difference with something completely new at inference time, that was not in the training data, or not used by RL training, is that while it will be able to use it to some extent, it has not been taught via RL how to best use in a reasoning chain.

The problem is that LLMs don't really have the generic ability to reason, so they instead need to fake it by either:

1) Fine tuning on reasoning data (very specific)

2) RL training (more generalizable, especially for math/coding)

3) Prompting that encourages "keep on going" "tree of thoughts" exploration where they may discover a reasoning chain largely by luck

Demis Hassabis has talked about combining LLMs with search (cf systems like AlphaGo) which sounds like it might be useful for math research.