Local ML inference benchmark: PyTorch vs. llama.cpp vs. the Rust ecosystem

Local ML inference benchmark: PyTorch vs. llama.cpp vs. the Rust ecosystem(kvark.github.io)

2 points by kvark 27 days ago | 1 comment

kvark 27 days ago |

Author of the benchmark here, as well as Meganeura - one of the tested frameworks. Results are constantly being refined, gaps are filled, but it's hard to get more hardware coverage by myself. Would appreciate someone running it on Intel Arc GPUs, both discreet and integrated. Happy to answer questions about methodology!

kvark 27 days ago |

Author of the benchmark here, as well as Meganeura - one of the tested frameworks. Results are constantly being refined, gaps are filled, but it's hard to get more hardware coverage by myself. Would appreciate someone running it on Intel Arc GPUs, both discreet and integrated. Happy to answer questions about methodology!