This device is selling for $1,999. In the product page on their website, they show the device plugged into a MacBook Pro.
A MacBook Pro already has excellent ability to run local LLMs, so why wouldn't someone just spend that $1,999 on maxing out their MacBook specs (M5 Max with extra RAM) instead?
Or why not buy a Mac Mini with 48GB of RAM for $2300?
Because this device has 80 GB, which allows running larger models (which is the whole point of the device).
AFAIK, the next significant step in terms of quality above 32 GB is 256 GB, but 80 GB still buys convenience.
> A MacBook Pro already has excellent ability to run local LLMs, so why wouldn't someone just spend that $1,999 on maxing out their MacBook specs (M5 Max with extra RAM) instead?
80 GB extra (48 → 128) cost as much as this device, but the base MBP is very expensive already (4000+$). This device is essentially a "cheap" addon for an existing system.
Perhaps they don't want a new/upgraded machine or prefer a more portable solution.
- CIX P1/CD8180 + 32 GB RAM with 30 TOPS integrated NPU
- Houmo M50 + 48 GB RAM with 160 TOPS INT8
Anyone know more?
In practice you are better off:
- if youre in the mac ecosystem using 2k and upgrading to the ultra class chip next time you want to buy
- if youre into windows wait for the next ryzen ai max chip slated for next year.
- if you want desktop consider gx10 given that you need a computer to use this anyway
In retrospect it is crazy how good the m1 max series is given the amount of ram and memory bandwidth.
> SoC (Armv9.2
> CPU + NPU) 30TOPS +
> dNPU 160TOPS