https://github.com/jwlaboratory/bite-the-bullet
Predictive Speculative KV Replication for Bursty LLM Inference(jwlabs.vercel.app)15 points by shreybirmiwal 3 hours ago | 1 commenthttps://github.com/jwlaboratory/bite-the-bullet