Ask HN: When is fine-tuning a small LLM worth it?

5 points by kooldeep7 4 hours ago | 6 comments

I'd be interested in hearing about your experiences. What kind of task did you use it for, what model did you train, and what were the results?

Feel free to share examples of what you've tried.

ogou 2 hours ago |

I fine-tuned a model for my first AI web app back in 2019. It created plausible reviews of artwork from prompts. They were totally hallucinated though and had nothing to do with actual images. It was interesting for language and bias analysis above all else. But, it was enough of a novelty that I got thousands of users. I have considered going through the whole process from scratch with Mistral to make a bespoke model and correlate it with a modern vision model. But, I don't want to spend any more money on this project. Hosting an unmonetized public facing AI app for 7 years was not cheap.

https://artreviewgenerator.com/

spottedmarley 3 hours ago |

I've trained two small models, one I attempted to train it to be an expert (decent?) hearts (card game) player, and the other I trained on dream interpretation texts, I wanted to see if a model could interpret dreams reasonably well if it were trained on all the literature in existence (that I could find anyway). The hearts experiment was a general failure because the game of hearts as played by humans just has too many strategies to consider. I trained it purely on games (~20,000 hands of hearts in the RL dataset I generated) and the trained model did about as good as any decently written procedural game logic would have. the dream analyst training was a bit more successful. I can hand the model a dream log and it will offer some interesting insights. are they accurate? hard to say, it's a very subjective type of training, but definitely a better experiment than the hearts expert training run. in both cases the biggest and most time intensive part is generating the training dataset. building the LoRa is the easy part.

I was using Qwen3.5:2b models for both, running on Dell Pro Max GB10 Cuda,128GB.

Vicmed13 3 hours ago |

I personally would only do it if I need it to solve a highly specialized problem that requires every inch of privacy I can get. Otherwise there are better options out there

minimaxir 4 hours ago |

It's worth it if a) you have a decent sample size of data for your problem and b) you have a cost-effective infra to host it.

Notably the latter is more of the bottleneck, particularly with the price race-to-zero with models such as GPT-6 Luna.

verdverm 3 hours ago | |

There are allegedly multi-tenant LoRA offerings in the works which would change the hosting-pricing constraints considerably. Keeping my fingers crossed they materialize

carlos_rpn 3 hours ago |

Maybe when you want to learn the process for when/if you ever need to fine-tune a larger model?

It's faster to make iterate when you're toying around with a 1B model than a 27B one.