LLM Microserving: a new RISC-style approach to design LLM serving API

4 points by jinhongyii 1 year ago | 1 comment

crowwork 1 year ago |

Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python

crowwork 1 year ago |

Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python