I'm interested to see if people are fine-tuning the larger LLMs just for kicks or if there are situation where it makes sense (not including high-stakes decision making like medical/legal) and it is computationally efficient?

Smaller models that are cheap to host, sure, but in which cases does fine-tune a larger model (and host it) really shine? Oppose to just using RAG and a closed-source API.

Perhaps it makes sense if it is serving a huge customer base, and the tone of voice needs to be different, but the question is how much work it is to train it and if it is worth it.

I'm not against fine-tuning but curious what the actual use cases are, where it makes economic sense and how successful people/organisations have been.