In-browser LLM inference engine with WebGPU and OpenAI API | Dark Hacker News

Dark Hacker News

new|best|ask|show|jobs

Built with the HackerNews API. Developed by LJT.AI

Note: The HackerNews API provides up to 500 stories per category

In-browser LLM inference engine with WebGPU and OpenAI API | Dark Hacker News

In-browser LLM inference engine with WebGPU and OpenAI API(blog.mlc.ai)

16 points by CharlieRuan 2 years ago | 4 comments

crowwork 2 years ago |

Comes with ability to do full structured generation with json schema

also a in-browser demo https://chat.webllm.ai/

leshenj15 2 years ago |

So cool. No more privacy concerns!

ruihangl 2 years ago |

It's good to see that it contains the standard OpenAI API interface in JavaScript which looks very convenient.

cyx6 2 years ago |

AMAZING!