I wonder, though. Why Ubuntu? Why not Debian?
With all due respect to Cannonical, Ubuntu is not really suitable. It is not aimed at developers
Unless it has changed since I left it in a fury, it takes too much control away from you with the Snap system.
docker-compose with a load balancer (traefik) is fairly straightforward and awesome. the TLS setup is nice but I wildcard that and just run certgen myself.
The main thing I think that's missing is some sort of authentication or zero trust system, maybe vpn tunnel provisioner. Most services I self host I do not want to be made public due to security concerns.
I'm going to have to look into this pterm thing.
I now run more than one app into one single VPS.
Very cool stack.
But does anyone have a resource or link that explains the idea to make a service which OP shared here?
Because frankly, I'd feel lost reading the code from one file at a time without knowing where to start.
Plus it's written in Go which I have I am not familiar with.
1) I see Kamal was an inspiration; care to explain what differs from it? I'm still rocking custom Ansible playbooks, but I was planning on checking out Kamal after version 2 is released soon (I think alongside Rails 8).
2) I see databases are in your roadmap, and that's great.
One feature that IMHO would be game changer for tools like this (and are lacking even in paid services like Hatchbox.io, which is overall great) is streaming replication of databases.
Even for side projects, a periodic SQL dump stored in S3 is generally not enough nowadays, and any project that gains traction will need to implement some sort of streaming backup, like Litestream (for SQLite) or Barman with streaming backup (for Postgres).
If I may suggest this feature, having this tool to provision a Barman server in a different VPS, and automate the process of having Postgres stream to it would be game changer.
One barman server can actually accommodate multiple database backups, so N projects could do streaming backup to one single barman server.
Of course, there would need to be a way to monitor if the streaming is working correctly, and maybe even help the user with the restoration process. But that effectively brings RTO down to near 0 (so no data loss) and can even allow point in time restoration.
2) yes yes yes! I really like litestream. Also backup is one of those critical but annoying thing that Sidekick is meant to take care of for you. I'll look into Bearman. My vision is like we would have one command for most popular db types and it would use stubs to configure everything the right way. Need to sort out docker-compose support first though...
I'll concede there's probably a little more hands on work doing things this way, but I do like having a good grip on how things are working rather than leaning on a convenient tool. Maybe you could convince me Sidekick has more advantages?
I'd also not want to have cloudflare as an extra company to trust, point of failure and configuration to manage.
But isn't this a little too tied to Cloudflare?
Caddy as a reverse proxy on that VPS would also give us free HTTPS. The downside is less security because no CF tunneling.
1. nginx + letsencrypt
2. forward based on host + path to the appropriate local docker
3. run each thing in the docker container
4. put Cloudflare in front in proxy DNS mode and with caching enabled
Your thing is obviously better! Thank you.
How do you run the containers on your VPS tho? You could still use Sidekick for that!
I think your setup is one step up in security from Sidekick nonetheless. A lot more work it seems too
Also, all of these tools have great documentation on getting up and running, but SIGNIFICANTLY LESS INFO ON HOW TO MAINTAIN OVER THE LONG TERM. If I was going to start using a tool like Sidekick, Kamal, or Dokku I would want clear answers to the following:
- How do I keep my VPS host up and running with latest security updates? - How do I update to more recent versions of Docker? - How do I update services that maintain state (eg. update to a new Postgres version) - How do I seamlessly migrate to a new host (perhaps as a way to solve the above)? - How should I manage and serve static resources & user media? (store on host or use cloud storage?) - How do I manage database migrations during an update, and how do I control that process to avoid downtime during an update?
I just spent an entire evening transferring a side project to a new VPS because I needed to update Postgres. The ideal self-hosting solution would make that a 20 min task.
One thing I’ve noticed is the prevalence of Docker for this type of tool, or the larger self-managed PaaS tools. I totally get it, and it makes sense. I’m just slow to adapt. I’ve been so used to Go binary deployments for so long. But I also don’t really like tweaking Caddyfiles and futzing with systemd unit files, even though the pattern is familiar to me now. Been waffling on this for quite a while…
https://news.ycombinator.com/item?id=41358020
I wrote up my own experiences too (https://blog.notmyhostna.me/posts/selfhosting-with-dokku-and...) and I can only recommend it. It is ~3 commands to set up an app, and one push to deploy after that.
I'll definitely be trying it out, although I do have a pretty nice setup now which will be hard to pull away from. It's ansible driven, lets me dump a compose file in a directory, along with a backup and restore shell script, and deploys it out to my server (hetzner dedicated via server auction).
It's really nice that this handles TLS/SSL, that was a real pain for me as I've been using nginx and automating cerbot wasn't the most fun in the world. This looks a lot easier on that front!
Then it finds the compose file based on the app name. It templates in the domain name wherever needed in the compose file, and if it's meant to be public it'll setup a nginx config (which runs on the host, not in docker). If the folder with the compose file has a backup.sh and restore.sh it also copies those over, and sets up a cron for the backup schedule. It's less than 70 lines of yaml, plus some more for restart handlers.
The only bit that irks me is the initial tls/ssl setup. Certbot changes the nginx config to insert the various certificates, which then makes my original nginx config out of date. I really like nginx and have used it for a long time so feel comfortable with it, but I've been considering traefik and caddy for a while just to get around this.
Although another option for me is to use a cloudflare tunnel instead, and then ignoring certificate management altogether. This is really attractive because it also means I can close some ports. I'll have to find some time to play around with traefik and caddy first though!
I'm somewhat surprised not to see this more often. I'm guessing supporting multiple linux versions could get unwieldy, I focused on Ubuntu as my target.
Differences that I see.
* I modeled mine on-top of docker-plugins (these get installed during the bootstrapping process)
* I built a custom plugin for deploying which leveraged https://github.com/Wowu/docker-rollout for zero-downtime deployments
Your solution looks much simpler than mine. I started off modeling mine off fly.io CLI, which is much more verbose Go code. I'll likely continue to use mine, but for any future VPS I'll have to give this a try.
Here’s a bash script I posted a while back on a different thread that does similar thing if of interest for anyone. It’s probably less nice than op’s for ex it only works with digitalocean (which is great!) - but it’s simple small and mostly readable. also assumes docker - but all via compose, with some samples like nginx w auto-ssl via le.
Docker != app. Perhaps it'd be more accurate to say, "to host any Docker container"?
I mean on rootless containers, yes, a lot of apps that need access to the underlying system might not work, but they are usually system stuff, not the kind you want to host on a VPS anyway. But when running as root I can't think of many.
But even ignoring those, if I'm going to spend all the time needed to containerize everything in to Docker images myself, why wouldn't I just run the programs directly and not deal with the overhead and extra work?
Does this only support a single app?
Nice project but the claims (production ready? Load balance on a single server?) are a bit ridiculous.
Thank you for pointing this out. When I was looking to install caddy, I was specifically looking for something without using docker since my VPS is 1g / 1cpu and that is what I based my comment off. When was reading the sidekick docs seemed by running one command that it would first install sidekick and then install the cert/app all with one docker file but now I am not even sure about that.
Appreciate you pointing that out, now I am back into analysis paralysis on which one I should use
I would love for it to support docker-compose as some of my side projects needs a library in python but I like having my service be in go, so I will wrap the python library in a super simple service.
Overall this is awesome and I love the simplicity, with the world just full of serverless, AI and a bunch of other "stuff". Paralysis through analysis is really an issue and when you are just trying to create a service for yourself or an MVP, it can be a real hinderance.
I have been gravitating towards Taskfile to perform similar tasks to this. God speed to you and keep up the great work.
So you can do 'docker build -t localhost/whatever' and then 'docker run localhost/whatever'. Also worth checking out podman to more easily run everything rootless.
If all you need is to move images between hosts like you would files, you don't even need a registry (docker save/load).
i’m building https://www.plainweb.dev and i’m looking for the simplest way to deploy a plainweb/plainstack project.
looks like sidekick has the same spirit when it comes to simplicity.
in the plainstack docs i’ve been embracing fly.io, but reliability is an issue. and sqlite web apps (which is the core of plainstack) can’t have real zero downtime deployments, unless you count the proxy holding the pending request for 30 seconds while the fly machine is deployed.
i tried kamal but it felt like non-ruby and non-rails projects are second class citizens.
i was about to document deploying plainstack to dokku, but provisioning isn’t built-in.
my dream deployment tool would be dokku + provisioning & setup, sidekick looks very close to that.
definitely going to try this and maybe even have it in the blessed deploy path for plainstack if it works well!
I'll reach out on twitter
Considering the ease of setup the README purports, a few hours of dealing with this might save me a couple hundred bucks a month in service fees.
As a side note, any reason why you decided against using docker in swarm mode as it should have all these features already built it?
- install docker
- run docker swarm init
- create yaml that describes your stack (similar to docker-compose)
- run docker stack deploy
If you want to just run a single container, you can also do this with `docker service create image:tag`
If you legitimately need to run your software on multiple OSes in production, by all means, containerize it. But in 15 years I have never had a need to do that. I have a rock solid bash script that deploys and daemonizes an executable on a linux box, takes like 2 seconds to run, and saves me hours and hours of Dockery.
When we ran it on kubernets, without touching it, it broke itself in 3 years.
Docker is fantastic developement tool, I do see real value in it.
But kubernets and whole ecosystem? You must apply updates or your stuff will break one day.
Currently I am using docker with docker compose and GCR, it does make things very simply and easy to develop and it's also self documenting.
I believe fly.io uses that. Not sure if OP’s tool does that
It feels much more dangerous to have such a system instead in place and provide false sense of security. Users know best what kind of data they need to backup, where they want to back it up, if it needs to be encrypted or not, if it needs to be daily or weekly etc.
Most of these I checked don't, but a recent Ubuntu version is perfectly fine to use as-is.
> Is there any automation for creating networks of instances?
Not that I'm aware, it would also defeat the purpose of these tools a bit that are supposed to be simple. (Dokku is "just" a shell script).
I also wanted to be able to remove Dokku if needed and everything would continue to run as before. Both of these work very well with Dokku.
Best part is that I can just dump whole docker-compose.yml files in and it just works.
“One feature that IMHO would be game changer for tools like this (and are lacking even in paid services like Hatchbox.io, which is overall great) is streaming replication of databases.”
And then I mentioned that I believe fly.io has litestream support. I think it’s fairly relevant to the comment/thread.
Same experience in both places.
The app broke itself.
Now we don't use managed kubernets anymore.
EDIT: So apparently what used to be known as "Docker Swarm" has been posthumously renamed to "Swarm Classic"/"Classic Swarm" and is indeed dead, abandoned, and deprecated. The project currently known as "Docker Swarm" is a younger completely different project which appears actively maintained. "Classic" still has roughly twice the GH stars and forks compared to the new one. I can't be the only one who's dismissed the latter, assuming it to be the former. Very confusing naming and branding, they would probably have more way more users if they had not repurposed the name like this.
https://github.com/docker-archive/classicswarm
> Swarm Classic: a container clustering system. Not to be confused with Docker Swarm which is at https://github.com/docker/swarmkit
If you have your code stashed somewhere else than thats already backup.
At the end of the day if you use this tool I guess all you’d need to worry about (given the tool is stable and works obviously) would be apt upgrades of the OS and even that you can automate, and then just figure out your reboot strategy. For me, I don’t even want to deal with that, so I happily use fly.
It's always a balancing act between outsourcing your heavy lifting, and having to trust that party and depend on them.
I think the benefits of Cloudflares DoS protection are vastly oversold and absolutely unnecessary for 99% of businesses.
(I think the false positives where some users randomly get captchas are actually bad for business)
I'm sure there's something smarter I can do, like reading back the result afterwards or someting and altering my local file. But honestly, once nginx is configured for an application, I almost never touch the config again anyway.
I suspect I'm more likely to move everything over to cloudflare tunnels and ditch dealing with ssl locally altogether at this point.
Even though I agree with you in spirit. The chances that you will need it are slim.
So I think it comes down to personal preference. This is going to sound a bit silly, but to me, running things in VMs feels like living in an apartment. Containers feel more like living out of a hotel room.
I know how to maintain an apartment, more or less. I've been living in them my whole life. I know what kinds of things I generally should and should not mess with. I'm not averse to hotels by any means, but if I'm going to spend a lot of time in a place, I will pick the apartment, where I can put all of my cumulative apartment-dwelling hours to good use.
I might tweak the apartment metaphor because I think it's generous to imply that, like a hotel, Docker does everything for you. Maybe Dockerless development is like living in an apartment and working on a boat, while using Docker is like living and working on a houseboat.
There is one thing I definitely prefer Docker for, and that's running images that were created by someone else, when little to no configuration is required. For example, running Postgres locally can be nicer with Docker than without, especially if you need multiple Postgres versions. I use this workflow for proofs of concepts, trials, and the like.
I personally run a bunch of software I've written, as well as open source things. So for me docker makes everything significantly easier, and saves me installing a lot of rubbish I don't understand well.
I then booted into a new fresh clean debian environment, mounted my disks, and:
cd /opt/docker/configs; for i in *; do cd $i; docker-compose up -d; cd ..; done
Sure, I have a new set of problems, but they feel smaller.
Like, legit, this is the whole point of docker. Application/service dependencies are no longer tied to the server it is running on, mitigating the worst parts of dependency hell.
Although, in your case, I suppose your tolerance for dependency hell has been quite high ;)
I get what you are saying, but note a word of caution - kernel upgrades can break container runtimes: https://github.com/containers/podman/issues/10623.
At some point I moved individual containers to other machines and they work perfectly. VPS, NUC no matter what.
As a side note, it's pretty interesting TrueNas is heavily designed around the ZFS paradigm. Most people think of applications and time stamps to restore not data pools and the snapshot paradigm accompanying ZFS.
With the right layer of abstraction on top of snapshots one could have the cake you need it to as it is difficult for beginners to grasp ZFS.
https://utcc.utoronto.ca/~cks/space/blog/linux/UbuntuKernels...
Well, the people who host the people do. Is that not argument enough in favor of it?
There are a lot of non-obvious gotchas with ZFS, and a lot of knobs to turn to make it do what you want. Anecdotally, a coworker of mine set it up on his development machine back when Ubuntu was heavily promoting it for default installs. It worked well until one day his machine started randomly freezing for minutes multiple times a day... He traced the issue back to some improper snapshotting setup, then spend a couple of days trying to fix it before going back to ext4.
For the Postgres data use case in particular, I would be wary of interactions and probably require a lot of testing if we were to introduce it... Though it seems at least some people are having success with it (not exactly plug and play or cheap setup though): https://lackofimagination.org/2022/04/our-experience-with-po...
There are a ton of ZFS knobs, yes, but you don’t need most of them to have a safe and performant setup. Optimal, no, but good enough.
It’s been well-tested with DBs for years; Percona in particular is quite fond of it, with many employees writing blog posts on their experiences.
Addressing your argument directly though: you know that if you spin up a Postgres database for your app, you need to dump the database to disk to back it up (or if you wanna get fancy, you can do a delta from the last backup + a full backup periodically). Anytime a Postgres database exists, you know the steps you need to take to backup that service.
Same with persistent file storage on disk: if you have a directory of files, you need a snapshot of all of those files.
Each _service_ can know how to back itself up. If you tell a Dokku _app_ to back itself up, what you really mean is that each _service_ attached to that app should do whatever it needs to do to create a backup. Then, dokku only needs to collate all of the various backup outputs, include a copy of the git repository that drives the app, tar/zstd it, and write it to disk.
As you pointed out, the user should probably be able to control the backup cadence, where those backups are shipped off to, the retention period, whether or not they are encrypted, etc, but the actual mechanics of performing a backup aren't exactly rocket science. All of the user configurable values can have reasonable defaults too -- they can/should Just Work (tm). There's value in having that work OOTB even if the backups are just being written to disk on the actual Dokku machine somewhere.
I sort of see it the same way as "just install Linux Mint, it's great, you'll love it", but when something doesn't work, it's "oh yeah, just open the terminal and ________": opening the terminal is the _last_ thing people want to do if they just wanna read their email or whatever.
Until you decide to optimise for resources, and do crazy things like “one postgres instance, one influxdb instance” instead of “one instance per microservice”, and then you get back into hell pretty quick.
Winds me up how massive tiny applications become, and how my choices are to throw money (RAM) at the problem, or money (time) at the problem. I wonder when someone will do the math and prove that developer laziness is having a substantial drag on global efficiency. The aggregate cost bourn by users has to be orders of magnitude larger than the cost savings made by developers at this point.