What a waste of time.
Strata already does this with Qwen3.8-Flash-Next: https://github.com/Niko1221/Strata
Which seems sufficient for mortals who are not running gas town, but want a local model.
may I ask why?
oai luna runs at 60-70 toks/s; astra runs around 35-45 , are you just worried that your agentic work can't spread across a lot of parallel work?
Given that you'd have probably a lot more capacity just to get to required bandwidth it could be used to switch very quickly between models (as all stored there would essentially be "loaded")
Maybe next you can run an LLM off of some old MFM drives
Most of the tokens consumed by agents is reading stuff - docs, tool outputs, code etc. Previewing a content of a dumb single page of html is a few thousand tokens.
If you use claude or some other harness you can ask it how many cache writes/input tokens are there in avg session, and how much time it would take if preprocessing speed was just 60 tok/s.