Running DeepSeek V3 671B on M4 Mac Mini Cluster(blog.exolabs.net) |
Running DeepSeek V3 671B on M4 Mac Mini Cluster(blog.exolabs.net) |
https://x.com/exolabs/status/1872444906851229814
Only just considering now that Strix Halo could help fill this gap that Mac chips with their huge memory bandwidth enjoy. 256GB systems shouldn't be hard to build!!
MI300a APU seems not popular but for consumers, this mix of big CPU and GPU seems perhaps quite compelling!
Why are we comparing FP16 performance when you're inferencing INT4 quantized models? Seems like a misleading figure to compare with when it's not really even the performance you're measuring.