Show HN: Capy-flash – a 125B MoE model on two 16 GB consumer GPUs

Show HN: Capy-flash – a 125B MoE model on two 16 GB consumer GPUs(github.com)

7 points by Feyd 2 days ago | 2 comments

A Strata (https://github.com/Niko1221/Strata) challenger!

The first one I've seen (although I'm sure there are and will be more as more time progresses...)

It seems to require a bit more system memory (70GiB free RAM as opposed to 32GB) and two newer nVidia RTX 50 series graphics cards with 16GB (as opposed to a larger choice of multi-vendor older single graphics cards with a minimum of 12GB), but on the flip side, it does seem to make up for that with faster token generation speed...

Anyway, it seems to be worth watching...

Strata may have a worthy competitor here!

Feyd 1 day ago | |

Hi thx for the reply. Capy works on single card also, but yes the optimisations are done for double card and for nvfp4 nvidia format, that's native on blackwell, so this can also be used in newer datacenter gpus. On the right setups it's faster than strata