768GB Intel Optane DIMMs to run 1T-parameter LLM with single GPU at 4tps

24 points by walterbell 8 hours ago | 1 comment

lostmsu 3 hours ago |

The bottleneck in this setup is PCIe bus. You don't need optane to saturate it. 4 regular SSDs might do just fine.

lostmsu 3 hours ago |

The bottleneck in this setup is PCIe bus. You don't need optane to saturate it. 4 regular SSDs might do just fine.