My Homelab AI Dev Platform(rsgm.dev) |
My Homelab AI Dev Platform(rsgm.dev) |
I am also creating this and enjoyed the post and comments all going through the same thing :)
I had a conversation with my lawyer and I had “just one more question” that was going to take more than the time we had left in the current meeting. He said “schedule another 30 and let’s talk about that.”
Fair!
https://codeberg.org/dragonfyre13/forgejo-opencode
Still tinkering with it, but the gist is that I can invoke Opencode with /oc inside of an Forgejo issue, then it will come back with a PR for me to review.
> I’ll share my homelab setup soon. There are about a dozen docker compose stacks for the services that I manage.
That is probably neat, but before I read, how many thousands of dollars would I need to spend to acquire the RAM and GPUs needed to do something similar?
I still need to find the time to get into the Forgejo code and add that endpoint.
On the Podman side, I wrote a tool named Materia[1] for it, but there's also the wonderful Ansible quadlet role as well as Quadit and Orchess.
[0] https://github.com/kimdre/doco-cd
[1] https://primamateria.systems or https://github.com/stryan/materia
Is it a deployment automation platform where it can run a project’s docker services, with rollback and all?
Then, I said homelab AI, I thought it's an interesting post about local GPU setup (and I am really interested in this topic).. but no, just another hype post about how to use whatever-code...
I was also hoping to put out another post on my homelab setup, it has some neat stuff, but I haven't had a chance to finish it.
The biggest issue I've noticed is that the chat templates for open models are really hit or miss. The default Qwen3.6 chat template mostly works these days, but depending on your workload it may cause major issues. There are plenty of "fixed" chat templates on hugging face, but people report mixed success. It really seems to depend a lot on what the tool you're using expects.
But there is a different tool that is an API accessing CLI: https://codeberg.org/forgejo-contrib/forgejo-cli
I have 27b, 35B-A3B and a cpu backed gpt-oss configured and use them in parallel, checking if one is getting ratholed and adding context or manual fixes.
I had various other systems setup and commercial models but really don’t use them.
It may be too interactive for some people, but it is a good mix of fail fast and often the places qwen3.6 was failing was eventually problems with the frontier models.
And this is with the unsloth defaults and hardened llama.cpp podman containers.
I do sometimes load other models or honestly just feed things into google’s free agent. But that is rare and to be honest manually fixing is typically faster and less error prone