If multiple models can use cache representations for this kind of enrichment, the KV cache representations of different models must be somewhat compatible.
What stops us then from going a step further, and producing a model family where all models are "KV aligned", and each model can utilize the KV cache of other models directly?
So, an "expensive" reasoning model can use its full faculties to plan, but "delegate" simple subgoals to a smaller model. That smaller model can access the large model's intent directly, as rich KV cache representations - with no prefill recompute and no associated "handover" latency. Or, likewise, a "cheap" small model can generate a diminished but highly compact KV cache that the "expensive" model can then operate on - for example, for skimming a large file for shallow patterns.
Curious to know if anyone is aware of research trying what parent suggested?
A "local 8B VLA" that runs at high frequency can be sufficient for things like sensor fusion, plan execution and manipulation dynamics, but insufficient for advanced problem solving and in-depth "exception handling". So being able to ask a larger model for steering cues is going to be valuable for AI robots. And that does introduce latency and interface boundaries.
Not sure if the exact approach described there is the way to go, but I do expect something similar to show up in deployments eventually. If nothing else, having this one described does provide a point of comparison.
Seems like it could still help but also feels like one of those things where it becomes vastly more complex and difficult to debug.
https://news.ycombinator.com/item?id=47195212
I wouldn't have thought to use it for LLM-to-LLM communication, though
Which is not necessarily something the humans training a model would want re/ alignment.
I think this is similar to how Gemma 4 12B is implemented, but even then I don't think the single layer image embedding is "aware" of the context.