Ask HN: What can I do with a single R9700?

4 points by scott01 12 days ago | 8 comments

Hi HN,

I’m considering getting an R9700, budget only for one unit currently. Is there anything useful I can do with local LLMs within its 32G VRAM? I’m also thinking to try and learn building a simple inference engine. Any thoughts?

spottedmarley 12 days ago |

For learning it's all you need. Pick a good small model, I would recommend Qwen3.5 there are models ranging from 0.8b params (will run on a smart phone) all the way up to 397b params .. I run the 122b param model daily with 128GB vram and get 20+tk/sec. regardless of the model, you can learn all about how to host and harness a model with any size

scott01 12 days ago | |

With this amount of VRAM, do you happen to be using a Strix Halo or a Mac?

spottedmarley 12 days ago | | |

It's a Dell Pro Max GB10. I have two that I can bridge together to run larger models across the 256GB pool but I only bridge them when I am experimenting with larger models and they will run fine but at lower tk/s with really only marginal reasoning gains, so I tend to stick to medium models (122b) on one GB10 and I use the other to serve gen models for music, image and video gen.

rdx-amw 12 days ago |

Plenty - you could do both with an R9700.

Unless it's CUDA only, a fair amount of software now supports AMD ROCm. And if you're already wanting to write an inference engine, 32G will be more than fine.

I've had a R9700 and it'd run the lower quants (4-5 bit?) of Qwen 3.5 27B reasonably attached to Hermes Agent. Not quite a 3090, but still respectable speeds.

FloatArtifact 12 days ago | |

You say that as it's in the past, have you moved on to something else?

rdx-amw 12 days ago | | |

No, the R9700 I have got fried by a power supply fault, and GPU replacements are not always trivial.

Of the two affordable options for good 32G compute, the R9700 would be the best of the two (compared to the Arc Pro B70).