I built a page that tells you what AI model your laptop can run(localsotabenchmark.engineersf.dev) |
I built a page that tells you what AI model your laptop can run(localsotabenchmark.engineersf.dev) |
The app states "Small Models (1-3B)".
(Edit: now it came to my mind that Firefox has fingerprinting protection by default. But even after whitelisting this website, the result is the same)
Thought it was nifty so shared. Would love to see a real CS person's version of this and/or best prompt to build something like this accurately and keep it up to date as models advance and SOTA shifts.
I do appreciate that the "canirun" one
1) was also wrong 2) but let me pick my real card from the dropdown with results that seemed much more accurate.
Genuine question - is there literally any factor besides RAM and GPU (with VRAM info?)
1. I thought the table showing a bunch of models after my laptop specs would be models I could run on my laptop.
Model Tier Example Models Params RAM Needed Your Status
Top SOTA GPT-4o, Claude 3.5 Sonnet, Gemini Ultra 1T+ Datacenter —
Large Models Llama 3 70B, Mistral Large, Command R+ 70B 64 GB Usable
Mid Models Llama 3 8B, Mistral 7B, Phi-3 Medium 7-13B 16 GB Usable
Small Models Phi-3 Mini, Llama 3 2B, Gemma 2B 1-3B 4 GB Your Tier
2. I don't know what 'usable' means. How can these models be status:usable if they are not 'my tier'? I deduce 'usable' doesn't mean usable on my laptop but I don't know what 'usable' does mean.
3. Give me links to set up the best model I can.