I have 6x P4s, a Xeon E5 2696v3 (36 threads, 3.8ghz peak but all core turbo unlocked, so 6 cores at 3.8Ghz - about 8 cores at 3.5ghz, or all cores at 3.1ghz), 48GB DDR4, all fit into a micro atx case running on a 650W MSI psu. This gives me a virtual 48GB GPU (llama.cpp ftw) to backup that 48GB of RAM.

I typically see scores of at least 7-12t/s on 20-30B Q4KM size dense models, on a 32K/48K/64K context, adequate for modern inference.

The pain point is the prompt loading, it is far far slower, minutes not seconds, than modern tensor core 8GB 5060s (my other machine's 2x GPUs) but is quite similar in regular inference speed once it has loaded.

That said, even the old Radeon Pro stuff has gotten more expensive on eBay, so I'm not necessarily recommending cheap old server cards that need custom-printed fan shrouds to operate in a consumer PC. Probably better to buy the Radeon AI Pro R9700 for $1400, which will be faster, supported for many years, and has a fan already. Or, maybe even the Intel ARC B70 for $1000.

This is an interesting article though. Bookmarking since my dual e5-v4 system is unplugged until summer is over.

https://www.pugetsystems.com/pugetbench/creators/davinci-res...

https://www.pugetsystems.com/pugetbench/creators/premiere-pr...

In some cases it is better to have lower passmark scores:

https://www.videocardbenchmark.net/gpu.php?gpu=RTX+PRO+6000+...

Blender is heavily bottle-necked by ray-tracing and de-noising operations:

https://opendata.blender.org/benchmarks/query/?compute_type=...

One metric that isn't considered is VRAM, as some rendering pipelines still rely on composited baked-scenes to reduce each areas memory requirements.

In general, the $/performance unit will depend on what you are doing, but there is 1 more thing to consider... Old GPU use mystery binary BLOB drivers no longer maintained on modern kernels. You might get the software to work with a legacy Windows GPU driver, but the key takeaway concept here is "might". =3