Show HN: Code to run Gemini (Nano) locally on desktop/Chrome

Show HN: Code to run Gemini (Nano) locally on desktop/Chrome(github.com)

22 points by notnotrishi 1 year ago | 5 comments

Chrome Canary (nightly build for devs) now has Gemini LLM inbuilt. This is just some simple code/demo to enable and use that feature.

samspenc 1 year ago |

Huh, looking at the code, it's simpler than I thought, basically these are the 2 lines that enable the request-response functionality:

  const session = await window.ai.createTextSession();
  const response = await session.prompt(prompt);

notnotrishi 1 year ago | |

exactly! also, one can set parameters etc. if they'd like.

also, after this post, i was able to do some tinkering and run it outside chrome, which i think would be more useful. hope they release the weights and make it open

DownrightNifty 1 year ago | | |

You should share this with us! If you have a blog, that'd be a really interesting post.

DownrightNifty 1 year ago |

Instructions didn't work for me. Tried on Windows 10 and macOS 14.3 (x86) with Chrome Canary. The "On Device Model" component doesn't show up.

notnotrishi 1 year ago | |

yea, it's a bit tricky. this came in from google team today:

> To get back on track, make sure you're using Chrome version 128.0.6545.0 or later. If you want the most up-to-date version, try using Chrome Canary or Chrome dev channel.

this document should have the latest changes from Google: https://docs.google.com/document/d/1VG8HIyz361zGduWgNG7R_R8X...