Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scale(daft.ai)5 points by ykev 256 days ago | 1 comment