Benchmarking LLM Inference Back Ends: VLLM, LMDeploy, MLC-LLM, TRT-LLM, and TGI

12 points by sherlockxu 2 years ago | 2 comments

ssheng 2 years ago |

Hello! We're the authors of this blog post. Please let us know if there are other models and inference backends you'd like us to benchmark next.

wey-gu 2 years ago |

Great work

ssheng 2 years ago |

Hello! We're the authors of this blog post. Please let us know if there are other models and inference backends you'd like us to benchmark next.

wey-gu 2 years ago |

Great work