I'm assuming that doesn't actually exist yet, though, as I don't see anything about an implementation in the code that's been released.
But, it's a really interesting idea for a personal model. There's a risk of more AI psychosis if these things actually start "learning", but the value of it is also probably pretty big. I'm not sure I buy it will actually be able to self-improve, though. The best models are helping improve themselves, but the best models are considerably smarter and more capable than this one. I've asked models like Gemma 4 31B to help figure out training and synthesizing data, and it mostly fails on anything more than categorization and summarization. This little model is much dumber than that.
So, I'm skeptical, but maybe there's deterministic tooling that can assist and maybe it will be scoped tightly enough to just learn and update facts and not so much try to retrain the whole thing.
"please write 250 words on the etymology and history of the word schlong"
The actual origin of the word is from middle high German and Yiddish-speaking Ashkenazi Jewish communities.
For comparison qwen 3.6 35B A3B does perfect on this and will give a solid description of the word's real origins and how it has made it into casual profanity/vulgarity as used in US English, and even mentions specific stand-up comedians and famous public figures of specific ethnic/religious origin in the US NE who introduced it into wider use.
Ask it for something that's not a narrow niche scientific or technical field, but something that would be less common to make it into a 20B size model, and see just how it does.
chat test link: https://chat.deepgrove.ai/
LLMs are kind of a compact knowledge box of its training data and it's understandable it would not have information about every topic and in that case just do a web search or a proper tool invocation to get the data and then synthesize.
No one is going to compress human knowledge into few bits.
To me, the benefit of running small models is that they fit on your device exactly that you don't need any internet connection. It's all local and offline, so you can still consult for information in any scenario.
It's also something I've seen has great results with esoteric individual pieces of knowledge that works fine in a Q6 or Q8 quantized LLM but breaks down in a bad way at worse quantization.
Models in this size range should aim for correct tool calling and avoiding hallucinations, not universal knowledge. (apparently they don't.)
An LLM continues the prompt it is given. What is more likely to come after a question? An answer, not an "oh sorry I'm not sure". Sure, you could make the latter more likely, but then the model would be unusable. Larger models simply contain more answers, more ways to stumble into them, and a granular enough geography to stay on the trail.
I wonder if kids will do this to their parents.
Will give it a crack as a quick model in my clawlike.
The model is reliable at the semantic half. Give it the OCR text of a receipt and it correctly identifies the vendor and the date. What it does not do reliably is follow mechanical instructions. A user asked for dates formatted as TT-MM-JJJJ and got files literally named TT-MM-JJJJ, because it reproduced the format string instead of filling it in. Another asked for uppercase, and the model acknowledged the request in its reasoning and returned lowercase.
The failures were not consistent, which is worse than failing every time. You cannot tell users "this doesn't work", only "this works most of the time", and nobody accepts that from something touching their files.
What fixed it was moving the mechanical part out of the model entirely. The model decides what the document is about; ordinary deterministic code decides how the name is written. Every time I moved that line back toward the model, quality dropped.
Which is a version of your point: with a small model the win isn't making it smarter, it's shrinking what you make it responsible for.
I didn't notice the version difference when first reading the article! So this is a heads up to people like me.
Really basic stuff. But then again, the entire thing was running in <6GB of RAM.
But before anyone says 1-bit bonsai 27B beats anything, please actually run it and ask it some questions about topics you already know the answer to.
While Qwen 3.6 35B A3B in Q8 with full context capability (llama-server in no-mmap mode with 262k context will eat 47GB, so not comparable in size either) knows a great deal. The 35B-A3B can even translate multiple pages of English into Farsi and its Farsi output is not far off the quality of what Google Translate does.
I haven't tested something as badly quantized as 35B A3B Q2 which is somewhere around 12GB on disk. https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF
Looking at the chart on this website, Bonsai Qwen 3.6 27B has a lower average benchmark score than Qwen 3.5 35B-A3B (77.1 vs 82.9)
Very excited to see how it performs, I’ve been a bit skeptical of the efficacy of converting existing models - really cool to see one trained from scratch in the ternary format.
Also I love AI sites. Fancy font, plain serious style, we "introduce" rather than "release". It's an AI not an animal after all.
It's definitely not going to replace a larger frontier model, but it's worth keeping an eye on.
...from very unscientific casual vibes it does seem pretty good though considering the speed
Would also be curious what their search tool backend looks like - that too is very fast for very rapid multiple searches
I probably could make up a bit of an incorrect carrot cake recipe if asked on the spot, but with a Google search I can give you something far more robust.
Maybe we don't need a 'country of geniuses' in our pocket, but more a helpful assistant that can reasonably reason!
Super cool and a taste of what's to come with local AI becoming more accessible to low-end hardware.
20b parameters * 1.5 bits per parameter is just 30 billion bits, about 3.75gb
a full 20b fp16 is about 40GB.
I find it weird how a smaller model still produces decent text, except it bullshits all the way.
If you want it to know more things locally, you could hopefully distribute wikipedia or other appropriate large sources on a flash storage, properly pre-indexed, with a local tool API, and let it work its magic on that. Not least because that is as I understand it, considerably easier to incrementally update.
This is, without doubt, a very difficult target, not least because it is rather at odds with the way LLMs work on a crude level.
It seems strange to me to expect a small model to answer everything correctly when every device that they are being used on support networking and knowledge is constantly evolving.
Mind you, because I am not that interested in long-context, fire-and-forget tasks, I think the faster MoE model has more value generally.
It's also the aspect of LLMs that degrades fastest with quantisation. You can't reasonably expect accurate knowledge of everything in the world in a few gigabytes.
Which means that small models need to be conditioned to rely more heavily on tools to fetch accurate information, and ideally not try and generate facts purely based on their (extremely lossy) internal knowledge