ChipBenchmark: Open-Source Benchmarking for LLM Performance Across Hardware

ChipBenchmark: Open-Source Benchmarking for LLM Performance Across Hardware(chipbenchmark.com)

3 points by technoabsurdist 354 days ago | 2 comments

We just launched Chip Benchmark, an open-source tool for hardware-centric benchmarking of open-weight LLMs across accelerators like NVIDIA A100/H100/L40S and AMD MI300X. It measures throughput, latency, and time-to-first-token with transparent scripts and an interactive web dashboard—making apples-to-apples comparisons easier.

We're actively welcoming contributions, new hardware support, and benchmark requests.

Repo here: https://github.com/Herdora/chip-benchmark Dashboard: https://herdora.com/benchmark

Feedback and contributions welcome! We made it super easy to add other architectures by including the script we used for benchmarking.

technoabsurdist 354 days ago | |

^ We currently just have llama3.1-8b, so we'll be working on adding more models across more hardware options!