Dark Hacker News
new
|
best
|
ask
|
show
|
jobs
My $600 Mac Mini Runs a 35B AI Model | Dark Hacker News
My $600 Mac Mini Runs a 35B AI Model
(thoughts.jock.pl)
4 points
by
danebalia
80 days ago
| 3 comments
bigyabai
80 days ago
|
next
[−]
> The 35B Trick (Your SSD Is the New GPU Memory)
Wave "bye bye" to your write cycles.
RobMurray
80 days ago
|
parent
|
next
[−]
why? it's mostly reads. the weights are static.
bigyabai
80 days ago
|
root
|
parent
|
next
[−]
llama-cpp's process is, but macOS itself will swap hard when 10-14gb of memory is paged for LLM inference. Dense models especially would thrash zram.