Super easy deployment of additional apps, defined completely in one file (incl setup on host, backups, reverse proxy config, etc).
Never found a reason to migrate away. Swarm was already considered dead when I started using it in 2022[1], but the investment was so low and benefits so big, that it was the right choice for me. I think a lot of people are replicating swarm features with compose, losing a lot of time. But hey, to each their own choice!
Using traefik or caddy as proxy.
Docker context for remote access - over Internet or vpn, whatever.
Swarm-cronjob for scheduled things.
Labels for things that need to run in particular places.
So easy.
Personally, k8s is fine, but its an abstraction for building a service architecture, not the thing an end user (developer) should ever use. If you are in a big company and you are using helm or k8s yaml files to roll things out, your infra or platform teams have missed something out.. building the platform!
https://developer.hashicorp.com/nomad
Disclaimer: I used to work for HashiCorp
It's nice to get an easy question every once in a while.
even their follow up - Docker Compose vs Kubernetes.
Docker compose for me has been great - no complexity.
And the error handling was terrible. Most of these problems resulted in a Python stack trace in some docker-compose internals instead of a readable error message. Googling the stack trace usually lead to a description of the actual problem, but that's really not something that inspires confidence.
Very few separate ecosystem transfers are quite that frictionless.
If you want to test something that is between compose and k8s, check ring: https://github.com/kemeter/ring
Docker compose assumes all your services can reach each other over docker, which I find horribly insecure.
I separate all my services by user account at least, maybe even by VM, and I run them all in rootless podman containers. So it just doesn't fit my style, but I'm sure it works fine.
Things like a good security posture by default, health checks, drift detection, and port forwarding.
The only issue is the little downtime during deployments.
Ie you need a sysadmin. Oops, you fired them all 10 years ago when the agile devopsing became the best thing after the pumpkin latte.
What I like for my prototype projects is how easy is to use it with podman too.
Granted, its B2B Saas with not many users, maybe 100 concurrent.
80% of workloads dont need the complexity of Kubernetes and run fine with compose.
If you run more than one service/codebase, you might be better suited to using a proper container orchestration platform. Doesn't have to be Kubernetes. AWS ECS, GCP Cloud Run, Kamal are all modern options here.
If you run a single codebase in production, why are you even containerizing? Language ecosystems have done a phenomenal job of improving their dependency management since Docker was released. Python has uv. Go has modules. NodeJS has pnpm. Do you actually get benefit from containerizing if you're deploying to a single production host somewhere?
Some time ago I've written about my experiences using it in production https://nickjanetakis.com/blog/why-i-like-using-docker-compo.... Not just for my own projects but for $500 million dollar companies and more.
We have been using it for more than five years now. Before that, we had a legacy deployment model, and I do not remember a single major issue related to Docker Compose.
We use it for both staging and production environments. The same Docker image validated in staging is deployed to production. Never fails!
This is why nobody uses it. Cloud stuff has to be as baroque as possible.
https://github.com/daitangio/misterio
It works very well!
But docker compose can actually be very sufficient for what many projects actually need.
Granted I am a guy pushing for compose based localdevs and such but going further you often just cannot beat the simplicity of doing update QA or other CI/CD workloads in compose based projects. I have had dozens of projects where we replaced flaky slow and maintenance heavy pipelines with just docker compose up --build --wait in the past years. How come you say health checks are still broken?
Agree.
Plus there's the monitoring of the host that is always overlooked in articles. I've ended up chucking Monit on there to monitor disk usage et al, and also used it for monitoring compose too and restarting containers.
And then there's Healthchecks.io, and external uptime monitoring... the list goes on. Properly monitoring systems, even single server systems, is not simple.
While not built in k8s has at least velero and kasten. However they are only possible because of snapshots https://kubernetes.io/docs/concepts/storage/volume-snapshots... and kasten has a plugin like architecture (because of k8s ) that supports application specific backups. However I never found something like that for compose. And that is troublesome in bigger projects like sentry
Docker volumes (and bind mounts) however have the minor problem of being hard to get a consistent copy to without stopping the service. You can work around this by, e. G., having ZFS or btrfs as the underlying FS and making a snapshot there. Otherwise, your software (like PostgreSQL) might also have other online backup tooling.
It's because I like keeping my servers stateless when possible. It makes it easier to upgrade them in a zero downtime way later.
If your web server has your DB too, then you can't do zero downtime system upgrades. For example I would never upgrade Debian 12 to 13 on a live server. Instead, I'd make a new server with 13, get it all ready to go and tested and then when I'm ready flip over DNS or a floating IP address to the new server. This pattern works because both the old and new server can be writing to a database on a different server.
With all that said, if you were ok with 1 server, then yeah I'd for sure run it in Docker Compose.
For most DBs it's one or two paths in the container, and virtually all DBs vendors have a reference Docker Compose example somewhere showing volume config. I can't remember the last time I ever "natively" installed a DB personally!
It really comes down to YMMV... Sometimes for a singular app surface, it's easier to just use a compose file that includes the database. mailu/mailcow is a good example... you don't necessarily want to comingle email on the same server as other services.
That said, if you need to share a single DB or set of DBs across an applicaiton with several instances/deployments, then it makes much more sense for a central deployment. I almost never do my own host level install, instead relying on cloud hosting and mgt. The only real exception is MS-SQL on internal servers... MS-SQL in Docker is barely acceptable for dev, and missing a few key features you may actually want/need.
What if you can't by yourself objectively evaluate if turkey sandwich sounds good?
It's not a matter of giving a universal answer to whether docker compose in production is fine, but how to evaluate it. Which features or safeguards necessary for a healthy production environment you forfeit when choosing plain docker compose? What's the tradeoff?
Comments like this are apathetic and reduce the challenges of good software engineering to hopes and random chance.
K8s as small time is overkill for sure but make sure you don't fall into this trap. https://www.macchaffee.com/blog/2024/you-have-built-a-kubern...
What it gets you is a more powerful Docker Compose running on server that you can interact with via kubectl. No SSH, no custom scripts etc. Just kubectl and YAML.
To me, if there's generally fewer than 10 actual active users at any given time and/or you can easily tolerate 30-60m of down time now and then... I'd lean into the simpler option of docker-compose. While I generally think of compose as a dev tool first, it's definitely useful sometimes.
Probably needs a real generalist though which maybe doesn't include your average dev.
> docker compose pull && docker compose up -d is a fine command if you are SSH’d into the host. At customer scale—dozens of self-managed environments behind firewalls, each with its own change-control process—that manual process doesn’t scale.
No idea what this 'customer scale' operation is, but it seems like a pretty clear cut candidate for not using docker compose. I also don't think watchtower should be listed there, it's been archived and was never recommended for production usage anyways.
We just use ansible for this part.
Isn't that a Docker thing rather than Docker Compose though? There is a ton more caveats to add if we don't already assume the reader is familiar with the hard edges of Docker, seems the article only focuses on Docker Compose specifically, probably because it'd be very long otherwise :)
* Lack of a user-friendly way of managing a Docker Compose installation on a remote host. SSH-forwarding the docker socket is an option, but needs wrappers and discipline.
* Growing beyond one host (and not switching to something like Kubernetes) would normally mean migrating to Swarm, which is its own can of worms.
* Boilerplate needed to expose your services with TLS
Uncloud [1] fixed all those issues for me and is (mostly) Compose-compatible.
If you remember `docker machine`, this is basically the modern version of that.
I've been using portainer for years, it's decent.
[1]: https://du.nkel.dev/blog/2023-12-12_mastodon-docker-rootless...
Could i survive with 10 seconds of downtime, probably, but I'd really like if I could avoid it.
https://uncloud.run/docs/guides/deployments/rolling-deployme...
They've basically lost the war against Kubernetes but they could easily claim a lot of ground when it's just one more tweak you're adding to your docker-compose file as it scales.
if you use Caddy as your reverse proxy (instead of nginx for example which does not do this), when requests come in and your service is missing because it's being deployed, Caddy waits for a timeout before giving up. this means that visitors during the brief deploy period don't see errors - they just get a slightly longer wait, which often is not obvious depending on how long your service takes to boot.
Yes, you can deploy a Go binary easily with systemd. Could you reliably do this across a fleet of machines? Including managing its configuration, persistent storage, database, network setup, etc.? Maybe, just need Ansible or equivalent config management. What if it were multiple Go binaries? And what if some of them needed to scale up some days because they hit more traffic than the others?
And on and on. Yes, not everyone needs Kubernetes, Nomad or other advanced orchestrators. But comparing them to running a Go binary with systemd is an unfair comparison.
https://docs.podman.io/en/latest/markdown/podman-systemd.uni...
Service file lives in the mono repo where all 6 services live.
Makes it trivial to make changes and redeploy.
This section misses the one thing I was interested in: how do you avoid downtime in a deployment?
I like to write web applications with Perl and Mojolicious, and a deployment is just "hypnotoad app", and then hypnotoad gracefully starts up new worker processes to handle new requests and lets the other ones exit once they've finished handling their in-flight requests.
When I switched to Docker I found that there was no good way to handle this.
edit: thanks to next comment for referencing one
> This is the shape Distr lands on
“Lands on”? I like that less.
Seems like an ad for whatever "Distr" is though; I haven't run into any of these issues with Dokploy and everything's been running fine for months.
Docker/containers are great, especially for local development. But I feel the docker compose model quickly becomes a lot of messy brittle squeeze for little gain when multiple containers need to integrate.
Better then to just take the plunge for the "real deal" and set up a non-HA k8s/k3s cluster with the interactions between the workloads clearly specified.
In other words. I care care more about the interactions declaratively spelled out than the "scale to the moon" HA, auto-scaling, replicas or whatever people get sold on.
And LLMs make this even easier. If you love reviewing yaml manifests....
It's been a great way for us to make sure that developers and CI/CD get exactly the same build environment, mount-points, paths, network access, permissions, etc.
It's been a super solid tool overall, and I'm pretty happy with it. The only thing that would make our setup better would be if we could figure out how to go rootless/daemonless with it.
Haven't used it in a while but this thing is also interesting--it supports a bunch of different ways to spin up k8s https://github.com/tilt-dev/ctlptl
I ran docker compose in development a lot. Just an easy way to turn on / off 5 different services at once for a project. Over time this was filling up my machine's storage (like 1 TB). Every few months I needed to run docker compose prune and see 600GB free up
https://docs.docker.com/engine/logging/drivers/journald/
I believe Podman can do something similar.
Is using Docker/Compose "just" as the layer for installing & managing runtime environment and services correct? Especially for languages like PHP?
I.e. am I holding it wrong if I run my "build" processes (npm, composer, etc) on the server at deploy time same as I would without containers? In that sense Docker Composer becomes more like Ansible for me - the tool I use to build the environment, not the entire app.
For the purpose of my question, let's assume I'm building normal CRUD services that can go a little tall or a little wide on servers without caring about hyper scale.
More to the point, there is no objectively right answer of what stack you should use. There are plenty of objectively wrong answers, but compose isn't one of them.
*shudder*
There’s a reason articles like this exist. Things change.
journald will help with logs, and the pull policy[1] helps with mutable tags. What help do you need with "orphan containers"?
[0]: https://docs.podman.io/en/latest/markdown/podman-quadlet.1.h...
[1]: https://docs.podman.io/en/latest/markdown/podman-image.unit....
You shouldn't be using podman compose. It's flimsy and doesn't work very well (at least it was last time I used it prior to Podman v3), and I'm pretty sure it doesn't have Red Hat's direct support.
Instead, activate Podman's Docker API compatibility socket, and simply set your `DOCKER_HOST` env var to that socket, and from there you can use your general docker client commands such as `docker`, `docker compose` and anything else that uses the Docker API. There are very few things that don't work with this, and the few things that don't are advanced setups.
For what it's worth, podman has also a thin wrapper (podman compose) which executes `docker-compose` or the old `podman-compose`. The docs should explain which it picks.
Note:
- `podman-compose` is an early attempt at remaking `docker-compose` v1 but for Podman. This used parsed the compose config and converts them to podman commands, and executes it.
- Later Podman wrote a Docker compatible socket instead, which can work with most docker clis that accept a `DOCKER_HOST` argument, including `docker` and `docker-compose` (both v1 and v2)
- `podman compose` is a thin wrapper that automatically selects `docker-compose` or `podman-compose` depending on which is installed.
Generally all you need is podman, docker-compose (the v2 binary), and that's it. From there you can use `podman` and/or `podman compose`.
Moreover, podman-compose integrates really easily with systemd. You can create a service by just running "podman-compose systemd" and following the prompt! No quadlet nonsense required.
https://docs.podman.io/en/latest/markdown/podman-systemd.uni...
Having your whole application with its containers, volumes, and networks all defined together in one easy-to-read YAML file is a way better experience. Deployment is two steps: 1. `git clone foo` 2. `docker compose up -d`. You can see the state of the application containers with `docker compose ps`. You can run multiple compose applications on the same host and manage them separately by putting them in different directories.
With quadlets, you delegate everything to systemd. You have to break the configuration up into a bunch of tiny unit files and then separately copy them to /etc or a dedicated user's dotfiles. An application with a handful of containers and multiple networks/volumes/etc can spiral into a dozen unit files. Good luck SSH'ing into an unfamiliar system and understanding at a glance what it's doing. It is far more annoying to predictably deploy and tightly couples your application configuration to the host system configuration. (Even moreso if you created dedicated users for each application, which I understand is the recommended solution.)
If I'm just holding it wrong and there exists some better tooling to manage podman in prod that I don't know about, I'm happy to hear about it.
* Podman fails to build a 16GB container image (after 30 minutes of downloading dependencies) despite having 90GB free out of a 200GB podman virtual machine
* Podman machine will, for reasons I don't understand, create a filesystem in a block device with wildly different sizes, and it seems like it's just random
* Pushing podman images to a container image registry via the Podman Desktop UI gives no indication that it's doing anything or even recognized the "push image" click, a success or error notification _might_ appear several or tens of minutes later or possibly not at all
* Starting a podman machine might work, but it fails ~75% of the time with not-particularly-exotic options (a bunch of ram and disk) and very cryptic error messages, frequently telling me to file bug tickets (I have)
* Podman Desktop won't let me create a podman machine with more than 44GB of disk, but the podman machine CLI won't let me create a machine with fewer than 100GB (IIRC--it's some number larger than 44, in any case)
Apart from the container image being absurdly large (Python developers love massive packages, I guess), I'm not doing anything exotic.
I always felt it the other way around: docker compose files are weird blobs of YAML that I have to hunt down the location of or parse their under-speced labels to find the location of. I can't make them depend on any non-container services[0], the break my firewall rules[1], and I have to use a whole mess of bespoke tooling just to do normal start/stop/restart operations with them instead of using the same commands I use for literally any other service.
> With quadlets, you delegate everything to systemd. You have to break the configuration up into a bunch of tiny unit files and then separately copy them to /etc or a dedicated user's dotfiles.
The nice thing about quadlets is exactly that, they integrate with systemd and by extension the rest of the system. I don't have to think about `webapp.container` as a "Docker container" I can think of it as just `webapp.service`, like any other piece of software I would install and run. All the related files are in one of the well-speced file locations that follow the same hierarchy as anything else on the system (user -> etc -> /usr), optionally grouped in folders[2].
> Good luck SSH'ing into an unfamiliar system and understanding at a glance what it's doing.
Just use the same tools you'd use on any other systemd system: `systemctl list-units`, `systemctl status`, etc. Versus having to hunt down compose files either manually or by parsing the under-specified labels on the containers.
> (Even moreso if you created dedicated users for each application, which I understand is the recommended solution.)
TBH I've rarely seen this advice. Most people I know just run it as root (which is what I do) or as a `podman` user. But even in this situation it should be pretty easy to figure out whats' running, as you know it's all running as one user and is hard-namespaced to only rely on resources available in that account.
> If I'm just holding it wrong and there exists some better tooling to manage podman in prod that I don't know about, I'm happy to hear about it.
Quadlets are just files that created systemd services, so basically any configuration management or deployment tool will manage them fine. Ansible has a dedicated Quadlet role that works pretty well, or just git clones+`systemctl start`. This would probably be the recommended way if you're not using k8s/etc.
Alternatively, you can just `git clone /etc/containers/systemd/`, `systemctl start container` like with docker compose. If you're running multiple containers, either refer to them with `Wants=`/etc in the Quadlet files, create a `.target` file that references them all, or put them all in a `.pod` and start the pod. I think this is the part were most people stumble though: when you're used to treating containerized software as a separate kind of "thing" it's a little weird to go back to treating it like normal services.
I've been writing something to help with deploying quadlets GitOps-style[3] that will hopefully fill the "more than one server but less than kubernetes" deployment gap.
[0] Unless I wrap the compose steps in a systemd unit, at which point now I have two problems.
[1] Caveat, this has probably gotten better overall but I still run into compose-related firewall issues about once or twice a year
[2] The newer versions of Podman also support `.quadlets` files, that merge all the quadlets into one file.
[3] https://github.com/stryan/materia . There's also https://github.com/orches-team/orches and https://github.com/ubiquitous-factory/quadit
If some BSD would support OCI containers, I would run my apps on BSD.
By the way most docker based implementations do not actually need the userland proxy docker runs automatically. Disable it in /etc/docker/daemon.js
{
"userland-proxy": false
}Like, if that works for you, more power to you. But that is a lot of moving parts in exchange for using a tool whose value prop is that it doesn't have many.
Also you don't even need the loopback address if the traffic is between one container and another, just a bridge network is fine.
For forward chain we set docker network ranges to route between themselves and only services actually used in containers. Allow container outgoing connections to our DNS servers, centralized HTTP proxy server and monitoring - nothing else containers are allowed to route to.
And for output is similar, only allow our DNS servers, NTP, HTTP proxy, centralized rsyslog where everything goes and zabbix monitoring server and a few icmp types - nothing else gets out and is logged.
With the advent of these supply chain attacks we read about often here it's just a matter of time some container is compromised and this seems like only viable way to at least somehow limit impact when such an event occurs.
The only modification is that I pin containers to an IPv4 address so I can limit the forward rule to that address.
We previously built a package manager for Kubernetes to abstract it in the simplest way possible `glasskube install app` but we failed because every abstraction needs to follow a "convention over configuration" pattern at some point. Also, we weren't able to monetize a package manager.
With Distr (https://github.com/distr-sh/distr), we have actually been able to help companies not only package but distribute and either manage or give their customers a way to self-manage applications. Our customers are able to land on-premises contracts at enterprises way faster than before, which is also a clear ROI for paying for Distr.
So, I don't think that you can get the flexibility of a distributed application orchestrator with a simple declarative YAML file if your target environments are diverse.
Just use k8s and follow similar patterns is the conclusion I've arrived at personally.
The inputs (values) are yaml so you can make it look exactly like a Docker Compose file if you want (wouldn't be surprised if there's some charts floating around that do that)
The simplest way to start probably would just be to create a custom Helm chart and define what you want in `values.yaml`.
It's perfectly fine, as long as you accept the risks and downsides. Your IP can get ratelimited for Docker Hub. The build process can exhaust resources on the host. Your server probably needs access to internal dev dependencies repository, thus, needs credentials it would not need otherwise. Many small things like that. The advantage is simplicity, and it's often worth the risk.
How? What I'm describing is using Docker less.
> The build process can exhaust resources on the host
Maybe, but I've yet to have a host where that's the case for usual CRUD fare.
> The advantage is simplicity, and it's often worth the risk.
That's basically what I'm evaluating for here.
For bog standard LAMP or similar stack applications, I've not understood the advantage of going through the build-image-then-pull-on-host rigmarole. There's more layers involved there than something like provisioning with Ansible and just having a deploy script to run the usual suspects.
But I have seen that done fairly often, hence was wondering what the point was.
Also adds moving parts to your deploy which increases risk/introduces more failure modes.
Couple things that come to mind
- disk space exhaustion during build
- I/o exhaustion esp with package managers that have lots of small files (npm)
However, on the small/hobby end I don't think it's a huge concern.
> disk, i/o exhaustion
This is why I mentioned specifically for ecosystems like PHP, which are interpreted. I'm specifically asking for that use case.
I'm not building binaries, my "build" steps are actually deployment steps (npm build, composer install, etc) that I'd be running in exactly the same way on the host. The image I'm deploying by definition also contains my source code because I'm not deploying anything compiled.
"Builds" are the same as deploys, so when working with server(s) instead of larger scale deployments, I'm not seeing the benefit of the whole "build image, pull on server" pipeline when I can just ditch the registry and added layers by doing those steps on the server as I would normally in other kinds of scenarios.
But I have seen this in action, which is why I'm wondering if I'm missing something.
The clearer benefit to me seems to be in this scenario to use it as a fast environment provisioning tool.
Some of my concerns with compose aren’t purely technical. It makes it easier to lean on local state like volumes, bind mounts, and large .env files. Similar mechanisms exist in kubernetes, but the additional setup tends to force a bit more thought about whether they’re actually needed or just a shortcut.
On the health check side, they exist, but compose doesn’t fully act on them, that's the part that is missing. There’s no built in remediation or orchestration behavior tied to health status, which is why things like https://github.com/willfarrell/docker-autoheal exist. It’s something that was never fully carried through in Docker itself.
https://www.macchaffee.com/blog/2024/you-have-built-a-kubern...
Your entire original comment looks like just an opportunity to be snarky. It's a longer version of "whatever", which you can literally throw around as an answer to anything.
In case you were curious, the subheading of the article already answers the question posed by the title:
> Yes, plain Docker Compose can still run production workloads in 2026—if you close the operational gaps it leaves: cleanup, healing, image pinning, socket security, and updates.
There are workarounds to make ipv4 work, but they complicate the system and make it more fragile.
I don't think there's a good one size fits all answer to whether hosting in Compose or RDS is right for you or a given project.
In my setups it just was easier to use fixed paths (or relative to project dir) from a permissions management perspective. Backup tools did not always have to/should run as root which is helpful on machines providing multiple distinct services.
Putting borg or a similar tool in a container that is part of the compose manifest file can also help. I haven't seen this used in practice though yet.
How would you upgrade the server running the database?
Using a managed database solves this problem, so there's that an option.
If you self host your DB, if the data is on block storage you can at least spin up a new instance and connect that storage device onto the new instance with a short period of downtime. This is usually a satisfactory level of downtime for an event that doesn't happen too frequently.
What I like about the above is it'll work with any database and avoids needing to even think about performing real-time or near real-time replication with multiple writers.
There's also the scary truth that there's a ton of stuff out there where compliance requirements aren't enforced. I'm not saying it's a good idea but you can choose not to upgrade too. This is a risk assessment you'd need to do. At the very least if you go down this route, please make sure your server doesn't even have a public IP address. If it's super locked down, that doesn't mean it's safe but you'll want to limit the number of attack vectors as much as you can.
And one nice thing about podman-compose is that it's ONE PYTHON FILE. You can just copy it into your source tree.
Caddy will even do fully automated valid TLS certificates for private IP ranges via DNS ACME challenge for free etc with renewals handled, so all my internal self-hosted sites have properly terminated TLS too, accessible by connected VPN clients.
It's funny that for many of us in our day job, we stand up private services behind a VPN all the time so only work clients can access it, but when self hosting don't bother with a simple wireguard/tailscale config etc.
That's what I answered for.
>I'm not building binaries
If you were, I would have added CPU to the list.
>my "build" steps are actually deployment steps (npm build, composer install, etc)
No, those are build steps. If you weren't using Docker, you would either run all those and shove in a zip/tarball or package into a deb/rpm, etc
>The image I'm deploying by definition also contains my source code
It doesn't contain .git or need credentials to your git/SCM
>I'm not seeing the benefit of the whole "build image, pull on server" pipeline when I can just ditch the registry and added layers by doing those steps on the server as I would normally in other kinds of scenarios
You don't need a registry--you can Docker save/load to push images directly to the server. Images buy you a versioned artifact with all the code-level dependencies baked in. Some maintainer yanks their package from npm? Who cares--you have a copy in your Docker image. Your new app version doesn't work? Edit 1 line to point back to the old image tag and rollback.
>> The build process can exhaust resources on the host
>Maybe, but I've yet to have a host where that's the case for usual CRUD fare.
When the build process completes, it tears down the overlayfs which causes everything to sync which leads to a big I/O spike. Depending on the server and amount of files, it might have no impact. However, I've seen build servers become completely unresponsive for 5+ minutes due to the I/O load when this happens. One place I worked, we had to switch our build servers to NVMe--the Docker container teardown caused spikes over 100k IOPs. Can't remember the exact details--it was React either React web front end or React Native mobile app.
>There's more layers involved there than something like provisioning with Ansible and just having a deploy script to run the usual suspects.
`docker save myimage:tag | gzip | ssh user@server 'gunzip | docker load'`
Not saying creating distributable artifacts is the de-facto answer, but I'd strongly consider whether it's really that much more complicated.
Fair enough, that buys a little bit of time to not break deployments I supose.
> When the build process completes, it tears down the overlayfs
Ah okay, I misunderstood you then - I was referring to Docker-less servers and my build steps running there, not building the images on the machine.
Thanks for the info!
Docker also commonly refers to Docker _images_ or Docker-esque container setups
Thinking about it a little further, though, I believe Rancher Desktop has come a long way and may be eating market share.
For a long time Docker was helpful and opened exposed ports on the firewall. So you wanted to access your redis ports locally and exposed it on the container? Now everything in there is accessible on the open internet.
I believe they've fixed it but I haven't used Docker in years so I wouldn't know.
This is not just for docker. There are other vms and lxc containers too.
I don't know if I'd necessarily call it brittle, though. You can use `helm template` and various linters to validate the generated yaml is correct (and use something like pre-commit to autorun)
Because if that's the case, I also don't care for baroque music that much.
Yes: okay try it
No: okay you don't have to try it
Unsure: okay you can read about it some more and decide if it sounds good to you
Im no fan of docker and podman by itself is a step up but orchestration headaches are enough to ruin that.
I don't understand what you're asking here. The answer to that is probably nothing. That is unless you want:
- systemd to manage your containers - You want to use K8s primitives (which are mostly compatible)
I'm unsure what the 3rd method is you're talking about. The nice thing about Podman's compose API is you don't have to change anything (mostly). You can point all your docker tooling to Podman's socket, and it'll (mostly) magically work.
* use systemd, red hat's favorite kitchen sink for handling everything from setting up sound services to mounting your home dir to logging so why not this too i guess.
* docker compose where i have to run a whole separate podman service to lie to docker compose about not actually being docker.
* podman compose which would be the obvious solution if it didnt just plain suck.
i also want to stay the hell away from quadlets or any other software which tries to make me use systemd more.
There is a theory which states that if ever anyone discovers exactly what Kubernetes is for and how it works, it will instantly disappear and be replaced by something even more bizarre and inexplicable.
This part is easy, Kubernetes is for your CV. /s
What happens if it errored on deployment or after that? you wanna write custom (bash? :D) hooks for that? What about upgrading your 'very vertically scalable' box? What if it doesn't come up after the upgrade? your downtime is suddenly hours, oops.
The k8s denial is strong and now rivals frontend frameworks denial. Never fails to amuse.
In our case, the answer is not "hope and bash". We deploy versioned images, use health checks, monitor the result, and keep rollback simple: redeploy the previous known-good image/config. Host upgrades are also treated as maintenance events, with backups and a recovery path, not as something Compose magically solves.
But I think there is an opposite mistake too: assuming every production system should be operated like a high-scale tech company.
Many production workloads are boring, predictable, and business-critical. They do not need aggressive autoscaling, multi-node orchestration, or constant traffic-spike handling. They need reliable deploys, backups, monitoring, health checks, and a clear rollback path.
That is where Compose can be a good fit: simple operational model, understood failure modes, low moving parts.
Kubernetes becomes much more compelling when you actually need automated failover, rolling deploys, autoscaling, multi-node scheduling, and stronger deployment primitives.
Not needing Kubernetes is not necessarily denial, it is just choosing the complexity budget that matches the problem.
2-3 miniPCs, cloudflare, tailscale, and k3s can save (possibly tens of) thousands on SaaS products, and would probably scale you to a company of dozens AND host your product.
I agree with gear54us and upvoted their comment, but I also understand what the author of the root comment is saying.
I have also delivered systems using Docker Compose that are actually running in production. The point I want to make is that people may define “production” differently depending on the number of active users, operational requirements, and risk level.
To me, this debate feels similar to the broader monolith vs. microservices debate.
Seems reasonable to assume these are serious production environments, no?!
Internally, though, they wanted to self-host a chat server, Apache airflow, Overleaf for collaborative editing of research proposals, three separate Git servers, a container registry, many other things, all with extremely strict multi-tenancy isolation requirements for storage and networking because they're handling customer data and their own customers audit them for it. That was a hell of a lot easier to do with Kubernetes than trying to figure out some giant universe of barely related technologies with vastly different APIs, having to buy specialized appliances for network and storage that probably also need their own control plane software hosted somewhere else.
But if you just look at "scale" as number of http requests a particular URL gets per some unit of time, the customer-facing sites have far greater scale. If you're trying to attribute revenue, beats me. They wouldn't sell anything without the customer-facing sites, but they wouldn't have anything to sell without the internal tooling. Solo web devs get into this tunnel vision view of ops because, to them, often the web site is the product. That's not the case for most businesses.
And, of course, they'd probably just use someone else's SaaS for tooling. But if you're in a heavily regulated space where that isn't possible and you have to self-host most of your business systems, then what?
Zero downtime server upgrades are easy. You could make a new server, ensure it's working in private and then adjust DNS or your floating IP address to point to the new server when you're happy. I've done this pattern hundreds of times over the years for doing system upgrades without interruption and safely. The only requirement is your servers are stateless but that's a good pattern in general.
I can't personally speak to what the limit of docker compose is, as I have only worked on the lower end of this: self hosting for personal use and for small internal services serving maybe 20 users.
Are you really going to try to get 4+ 9's of uptime for a small, one-off app? Do you really need to use a cloud distributed data store that only slows things down for no real gains in practice? Do you really think the cloud services are never down, and you're willing to spend a f*ck-ton of money to create a distributed app when historically an Access DB or VB6 app would have done the job?
I've moved applications deployed via compose pretty easily... compose down -t 30 then literally sftp the application to a backup location then to the new server which only needs the Docker Engine community stack installed.. then compose up -d ... tada! In terms of deployment, you can use github action runners if you want, or anything else... you can even do it by hand pretty easily.
You don’t need to live at the edge of new features. Do you upgrade your fridge and your oven every two months? It’s nice when you can have something running and not worry that the next update will break your software and/or your workflow.
To each their own, but this is the reason I advice newcomers to stay away from Debian based distro's. I don't intend a distro flamewar, it works perfect for `boring old and feature complete software´ like Dovecot.
To add: containers would alleviate a good part of these concerns, but the stupid thing here is that precisely that is broken for up-to-date podman workflows.
Systemd is a tool for managing services. Containers are services. Why require an entirely separate bespoke service manager when you're already running one?
> * docker compose where i have to run a whole separate podman service to lie to docker compose about not actually being docker.
This is the same system state as using docker compose with docker: you have a client program speaking to a backing daemon. Only difference here is the Podman service, being daemonless, only runs when needed (assuming you're setting up things the documented way by enabling the podman socket).
> * podman compose which would be the obvious solution if it didnt just plain suck.
Yeah I haven't had the best luck with it either. But part of the reason it's languished is that it makes more sense to just reimplement the Compose spec on the backend rather than re-invent the wheel and create a new compose client as well.
There's also the fourth option of writing Kubernetes yaml and applying that with `podman kube play`. Honestly this is probably closer to being the podman equivalent of docker compose but since it involves writing The Bad YAML (kubernetes) rather than The Good YAML (compose) most people don't use it.
It's a tool for user age verification that happens to be something you can use to manage services.
Did you miss my point about it being a filthy kitchen sink?
>This is the same system state as using docker compose with docker
One of the major selling points of podman is that you dont need a daemon. except maybe yes you do because podman compose sucks so toss that selling point in the trash.
This shit is also incredibly fiddly. Ever had "docker compose" and "docker-compose" do subtly different things which drive your team mate to pull their hair out? I have.
Podman should stop trying to piggyback off docker if it's trying to be an alternative.
>Yeah I haven't had the best luck with it either. But part of the reason it's languished is that it makes more sense to just reimplement the Compose spec on the backend
Personally I suspect it languished because Red Hat simply cant abide the idea that somebody out there might avoid using systemd for something.
They happily built a docker compose to quadlets converter but they cant bring themselves to make podman compose not be a piece of shit even though it wouldnt be a lot of work.
Good talk buddy.
> Did you miss my point about it being a filthy kitchen sink?
I suspect there's not really a point in responding to this since you've already made up your mind.
Nevertheless, yes I am aware the systemd project contains many modular components. Some of which are good (systemd-the-service-manager that is what I was referring to), some of them are bad, and some of them are just odd (still haven't wrapped my head around systemd-homed's purpose). Podman integrates with the systemd service manager, not the rest of the project, so I'm really not concerned about that: there is no point where I am unable to use quadlets because I don't have, say `systemd-timesyncd` installed.
On the gripping hand, Quadlets are just a systemd-generator so there's nothing stopping you from getting that exact same benefits of Quadlets with some other service manager. You'd just have to write that implementation (and probably your own bespoke service manager) and will probably miss out on some of the niceties systemd provides to anything it manages.
> One of the major selling points of podman is that you dont need a daemon. except maybe yes you do because podman compose sucks so toss that selling point in the trash.
You skipped the second part of my sentence where I reminded you that Podman is daemonless. There is no long-running Podman daemon/service/etc, it is spun up on demand and then stops when the action is done. Having a second process instance is not a daemon, and I'm not sure how you would have expected this to work otherwise.
> Ever had "docker compose" and "docker-compose" do subtly different things which drive your team mate to pull their hair out? I have.
..Take this up with docker?
> Personally I suspect it languished because Red Hat simply cant abide the idea that somebody out there might avoid using systemd for something. > They happily built a docker compose to quadlets converter but they cant bring themselves to make podman compose not be a piece of shit even though it wouldnt be a lot of work.
I don't think `podman-compose` was ever an official Red Hat project. I don't think there was every really much interest in ironing out all the corner cases, especially before compose was actually fully specced, and once Podman itself implemented the spec the interest has been drying up.
Assuming you're referring to podlet[0] for the latter, that was never a Red Hat project.
Lots of data, caching, web apps, background workers and lots of various API integrations. No fancy React front-end, no fancy crazy system architectures. Just a typical LAMP stack but running in Docker Compose, cranking away serving value to customers with very good uptime and a very low cloud cost relative to revenue. With that said, a managed database was involved but all of the web traffic was served by apps running through Docker Compose with a simple git push model of deployment that handled thousands of deployments over the years without much fuss.
I suspect you ignored the point because you didnt want to address the point.
My repeating it seems to only have highlighted your wish to continue avoiding it here.
>Nevertheless, yes I am aware the systemd project contains many modular components.
Another red herring. "Modular" really isnt the point here.
It's certainly one way to justify throwing even more shit in an already overloaded kitchen sink though.
>You skipped the second part of my sentence where I reminded you that Podman is daemonless.
No, you skipped the part where I acknowledged that it was daemonless-by-default but you actually DO need to run a podman daemon if you're using docker compose with podman.
>Take this up with docker?
They're not responsible for podman trying to piggyback on their tools.
outline a simple real world system to illustrate?
Get a few Beelink SER5 or SER9's, install Nextcloud to cover the files, document editing, communications (to save on Microsoft 365). Then you can have Gitea (and gitea actions) for your source code and building (skipping github enterprise), Harbor to host and scan your containers, frappe for HR, etc. Pretty much anything you pay enterprise rates for, you can self-host a version that will get your company from 1 to 100s with minimal extra work. If it's not on https://github.com/awesome-selfhosted/awesome-selfhosted, you can probably vibe code it in a couple hours.
I just started to run a k3s cluster with an almost enterprise grade software factory and a few (light) production workloads on a single cheap minipc.
For up to low hundreds of users I think you're better off just with 1 vertically scalable box for all the officey / web server workloads.
You mitigate the hardware failure stuff with a vendor contract where you can get someone on-site and overnight you parts, and by keeping things super boring. Volume replication is not boring, avoid at all costs. NAS or SAN if you have to but all disks in the main box for as long as you can.
For 20 person SME maybe a 2-bay Synology or similar, for a heavier company a low end 2U with hardware support. Proxmox under the OS for reduced worry snapshots, rollback, backup etc. Proxmox is there for operational flexibility, resist the temptation to create a network of VMs, you just need 1 CT or VM with all the workload inside it.
For container workloads on 1 host Portainer works as well as k8s IMHO, it gives you the key property you want - you can IaC everything declaratively with terraform + compose over an API.
Caveat that if CI gets heavy you might need to scale that out but you can keep it stateless.
I have several VMs in clouds with similar k3s architecture as yours and am wondering if there are any benefits to installing longhorn vs sticking to logical (postgres, mimir, whateveritis) replication instead.