Training a 3.8B LLM to 0.384 CORE for $998(hugovergnes.github.io) |
Training a 3.8B LLM to 0.384 CORE for $998(hugovergnes.github.io) |
LLMs are interesting in their own ways but as an engineer, this is a way to unlock a new way of building software.
I recently build a Claude-assisted Excel/CSV parser for a US based property management system (tax compliance). Uses Haiku and has a lot of deterministic code to extract column/row combinations to check known formats and finally handing out the headers to Haiku to give us a translation plan to our support columns.
These would eventually become part of the software, in a tiny LLM. The gap between training (such tiny LLMs) and inference will shrink. We can consult Claude for edge cases, create sample dataset and train a the tiny LLM on demand so we go to Claude less.
The tooling that a project needs is really important. Something I have been feeling as well. Not just in LLM building projects, but regular software projects that are LLM generated.
Stated too strongly, but I think this could be the model for education (some subjects anyway). Everything personalized to your learning goals, grounded in experiments that give a tight feedback loop and with a model that never gets tired of re-explaining something for the 10th time.
I have accepted two things that make me a happy engineer now: AGI is not here no matter what they say and LLMs are still very useful if one knows how to use them.
They are another layer of abstraction and like you said they do not tire. There is a lot of optimization needed so we can reduce wastage (running 1T+ LLMs for most work is wastage).
I’m with you. But what are we going to get? I think this goal sits in a funny place between knowledge and convenience.
On the one hand, tons of “products” promising this.
On the other hand, I’m sure we can find student works—sharing the code they created from a course or book. And I expect there will be gaps, niches filled by pro-coders who see a pro need and fill it.
What I don’t want to do is see the market for model training filled with whatever Microsoft thinks will make money.
Personally I find this a productive way to produce documents. Claude makes the first pass and I edit line by line. It’s a good understanding check. If I can’t reword something into my own voice it means I don’t understand it. Then have Claude loop again to fact check the rewrites and repeat until clear (or I disagree with Claude on all it’s remaining points).
That’s the same edit loop I used without LLMs, but I get speed in drafting up front (and fact/number pulling) and the check passes make the edits better.
The world of tiny LLMs is so interesting. It is unlocking novel ways to encode information. Why focus on the style of writing instead of the subject matter?
Because just as with human writing, a poor writing style obscures the topic where it should illuminate
I read some Dostoevsky recently and I found my speech to be substantially altered for a while, so it makes sense that LLM writing would have an influence.