Reasons to Drop Docker for Podman(developers.redhat.com) |
Reasons to Drop Docker for Podman(developers.redhat.com) |
1. You need a watchdog process anyways to handle stdout/stderr processing (to not block those pipes), namespace setup and teardown, etc. Why not have 1 daemon, rather than 1 daemon per container?
2. You need root to setup networking and various security controls (seccomp etc). Having a daemon lets an unprivileged user delegate to a privileged daemon. This was especially important in the early days when rootless containers and slipr4netns didn't exist or were immature.
3. You need to avoid contention between two "docker run" commands, such as avoiding downloading the same image twice, or running the same name twice, or so on. In-memory locks are really easy to get right and cheap
4. You need to be able to efficiently gather state for things like "docker ps" etc to work. A daemon lets you cache a lot of information very easily.
5. A daemon lets you provide a clean rest API to other components that want to "exec" or "list containers" or so on. Without a daemon, your API is just "exec this command, use stderr/stdout", which is worse, right? An API also helps immensely with "docker desktop" like stuff, where you want to run a client on the host, and run containers on a linux VM. I guess the daemonless answer there is "exec over ssh", which is slow, and also eww.
So, how does podman solve these? I'll number them the same:
1. Watchdog process per container (N daemons) plus centralized systemd daemon if you want to background a container, or for restarts to actually work.
2. slipr4netns, which is buggy and still less featureful than docker networking. It also just doesn't implement various security features that would require a privileged component to setup, and makes it harder to run root-ful containers.
3. Contention is avoided with an incredibly over-engineered filesystem locking scheme that has had a long history of deadlocks and bugs.
4. Again, the filesystem locking scheme and api. It's slow, can't cache well, and also has a long history of deadlocks
5. The API for podman is uh... "run a daemon" https://docs.podman.io/en/latest/markdown/podman-system-serv... lol. Except how does another app know if you ran the API because it's optional? This makes it so other applications that want to integrate with podman have to say "use podman and also run the podman daemon if you're not, and if you forget we won't auto-detect podman sorry".
In general, podman's answer to all the problems the daemon solves seem materially worse, and more overengineered. Except to the API bit, where their answer is to post all over the place "podman is daemonless", but then to package a daemon, which is just funny.
Having a daemon per container has this little advantage that if something manages to bring down one of the daemons, it won't bring down the whole shebang.
GitHub actions auto build new version releases for me so major versions become available as soon as they are released and I click the button
What did the author mean by Podman is not a replacement of Docker?
It works differently, and (in her opinion) in some ways better.
But you will likely have to change your workflow to use it.
Podman Desktop simply doesn’t work, on first run it loops forever on initializing stuff (I guess it tries to create the Podman machine but fails? No idea because it doesn’t say what’s wrong, nor where to look). So I tried Podman bare without Podman desktop and it’s not a lot better, the machine starts fine and I can run containers, but every time my computer wakes up from hibernate, the containers and the machine are stuck. I have to recreate the whole Podman machine from scratch.
I loved the idea of rootless but it doesn’t work on Mac. And I won’t believe I’m the only person having the exact same set of issues on 2 different MacBooks
> I loved the idea of rootless but it doesn't work on Mac
One clarification I think is worth making in case you weren't aware is that the "rootless" approach isn't really a factor for any Linux container runtime on MacOS since all the container solutions on MacOS run in a VM (since Linux containers rely on Linux kernel features). I.E. Docker Desktop, Podman Desktop, etc. can't run as root on MacOS because they rely on a user-level Linux VM.
Maybe I’m using the wrong term, but when installing Docker, you need root access, and not for Podman. Maybe I’m wrong but I don’t think it’s possible to install Docker if you’re not root on the machine?
I mean, I know that linux containers only run on linux (please ignore docker for windows which briefly did not need a vm, that is thing of the past now). I don't know if other people know that linux containers only run on linux.
> utilities that are supposed to work with Docker only work without Docker Desktop, or are unaware of Docker Desktop's VM and when installed claim there is no Docker on the host
> gargantuan difference between these implementations with and without the "desktop" Docker is internally significantly different, requiring completely different runtime configurations for 3rd party integration
It's not really that different though. Because docker has a daemon and API, anything that integrates with docker is supposed to talk to the "$DOCKER_HOST" environment variable to run docker containers, see if docker is running and what version, etc etc.
That should be identical whether "$DOCKER_HOST" is pointed at a unix socket on your host, or at a linux vm, or so on.
The implementation of those APIs should be identical.
Do you have an example of a specific utility? Did you not set "$DOCKER_HOST"?
Most of my colleagues don't know what a ssh public key is used for.
You're of course right that there's a huge diversity of programmers (in terms of experience, background, and extent to which they give a shit even if they're experienced and had access to the most helpful family background and best education a wealthy developed country can provide). But don't you kind of think that we should still encourage all programmers to see that _programmatic_ interfaces (in particular shell programming interfaces) offer so much power that they should be embraced and looked upon with enthusiasm where possible? And that we should not create static non-programmable GUI interfaces for programmers?
Good progress though and I'll revisit it again soon.
I think VirtioFS fixed most unbearable fs slowness.
But Rancher Desktop also got better and I just cannot understand why I would use Docker for Desktop instead of the Open Sourced Rancher Desktop.
Don't know why.
Colima on the other hand, is install-and-forget (so much so that I had to look up what it was called again). It uses Lima, which means you can do more with it it if you want.
What does cloud-native mean actually?
"Cloud native is the software approach of building, deploying, and managing modern applications in cloud computing environments."
An example of this would be how Podman Desktop includes tooling that allows you to easily deploy and manage pods in Kubernetes.
And use Skaffold or Tilt to manage it all.
This setup makes Docker Compose look like a toy.
Developers should stop with this impedence mismatch of using Docker Compose when developing and K8s in prod.
As a development tool it’s awesome, and having no centralized daemon is sure a boon.
I do use podman daily in the form of distrobox since both the steam deck and my desktop are immutable systems.
As much as it pains me to say this, I think nixos might be my next path.
The transition to Docker development is very poorly documented. I've taken two formal classes in Docker, read a book, and have half a dozen Docker projects done and delivered and this is the first time I've even heard one needs to manually set "$DOCKER_HOST". This industry is just a bunch of overly paid amateurs, blindly groping in a dark cave, a cave carved out of money.
Some background: I avoid most of the Enterprise Linux ecosystem (CentOS, RHEL, AlmaLinux, etc.) as my mentality to sysadmin is often quite different than the average RHEL lover. I run NixOS everywhere and supplement with Arch Linux or Ubuntu when I can't use NixOS. So I am in no way a part of the target demographic of RHEL or derivatives.
But, I actually agree with Red Hat's view on the RHEL clones[1]: they should start using CentOS Stream as their upstream base. CentOS Stream is upstream of RHEL, so if the RHEL-compatible distros start snapshotting from CentOS Stream, then all of the Enterprise Linux ecosystem will benefit from bug-fixes, improvements, etc. that they all contribute into CentOS Stream. Unfortunately, for the RHEL-clones, if they adopt this approach they wouldn't have bug-for-bug compatibility with major RHEL releases, and they would have to do more work in making an LTS off of CentOS Stream. But, everyone benefits (theoretically) in contributing to the same upstream codebase -- and I can't think of many instances where an application that works on RHEL wouldn't work on CentOS Stream or a downstream derivative.
AlmaLinux has chosen to rebase off of CentOS Stream[2] and I think it's the right choice. Rocky Linux has chosen to try to workaround Red Hat's removal of source availability by relying on loopholes to obtain RHEL source code via UBI containers and cloud images[3]. I can't imagine this (seemingly fragile) approach being sustainable and it feels counterintuitive to continue building a distro based on the work of a company whose mentality that you fundamentally disagree with.
Long-winded response, but I guess I don't love the FUD surrounding Red Hat. Red Hat's work benefits me as a Linux user with tooling like Network Manager, systemd and GNOME. I don't think Linux would be as serious of a contender in certain spheres without Red Hat's open source work that the entire Linux ecosystem benefits from.
[0] https://www.redhat.com/en/blog/furthering-evolution-centos-s...
[1] https://www.redhat.com/en/blog/red-hats-commitment-open-sour...
[2] https://almalinux.org/blog/future-of-almalinux/
[3] https://rockylinux.org/news/keeping-open-source-open/
EDIT(s):
Some minor grammar fixes.
Also, I would recommend looking at the the AlmaLinux link's footnote for ABI compatibility and what that means.
I was wasting more and more time on it. It was constantly crashing/hanging so one day I just said fuck it and switched back to Docker Desktop, which not only worked better than when I switched but seems to have improved quite a lot in the intervening time as well.
Colima on the other hand works pretty much seamlessly, but really works best on macOS Ventura because you can enable vz virtualization.
Edited to add: colima works seamlessly on the command line. So if you're command-line-averse, you'll need to learn some new commands and things won't be as convenient. I haven't tried rancher desktop yet.
Don't forget there is also Rancher Desktop[1]
It has been a long-standing issue but seems as though it is on its way to resolution: https://github.com/containers/podman-desktop/issues/1633.
I am waiting for a few months to make sure it is all sorted and will try again to see if it works then.
* podman with --network host won't expose the ports.
* If you suspend the laptop the time on the VM goes haywire.
I always found docker to be unreasonably buggy too so am not sure how I feel about this.
podman-desktop is pretty rough, but it gave me enough confidence to start using podman generally. podman-mac-helper solved a few docker compatibility problems I had in previous attempts in 2021 and 2022.
It seems like my problems getting started with podman just last week wete mainly due to the abundance of already outdated tutorials. Maybe those who are more active in this space can weigh-in, but fixes and common work arounds documented as recently as 6 months ago aren't necessary an longer? For me, podman works without nearly as much fuss as I remember.
I just verified running Testcontainers tests via CLI with those env vars you posted. YMMV
I wanted to have a separate network on a bridge, visible to the host, where IP addresses and DNS would be managed by a dedicated DHCP/DNS service (like what dnsmasq can do). Well imagine, unless you jump through a whole lot of hoops and use macvlan and whatnot, netavark just plain cannot do it.
I heard one could make a netavark plugin quite easily, but interfaces for DNS and IPAM are missing from the puzzle.
I don't use testcontainers myself, but it looks like as long as you are using Rancher Desktop >= 1.0.1 you should be just fine.[1]
[1] https://github.com/testcontainers/testcontainers-java/issues...
This link breaks down what permissions are used on MacOS.
> Maybe I’m using the wrong term
Typically, the meaningful piece with "rootless" Docker is that the daemon is not running as root.
When the Docker daemon is running as root on a Linux server, for example, anyone who can access the daemon (i.e. anyone in the "docker" group) has enough access to the system can do catastrophic damage with the access they have. For example, the docker daemon can mount any file on the host's filesystem (i.e. "-v /etc/shadow:/tmp/shadow"). With Docker running as root, anyone with access to the Docker daemon has the power to do almost anything to the system.
With rootless Docker, that issue is mitigated heavily because the Docker context is restricted to an unprivileged user context.
> but when installing Docker, you need root access, and not for Podman
According to Podman Desktop's docs, it asks for admin permission when installing on MacOS: https://podman-desktop.io/docs/Installation/macos-install
That being said, I don't personally see any security value added or removed by an installer process needing to elevate privileges. That's a one-time thing and likely should require admin privileges.
Where I worked before we didn’t have root access on our laptops, so we couldn’t install Docker.
I’ve switched company since, but my former coworkers were able to install Podman (not Podman Desktop) without root access.
Although, I would say we have definitely strayed far away from the typical definition/security benefits of "rootless" container runtimes. Usually the rootless container threat model accounts for containers or access to the runtime being weaponized -- it's not usually IT preventing you from installing apps. :)
Still, thanks for indulging this conversation.
(Also, I thought the only way to run Podman containers locally on MacOS was Podman Desktop -- has that changed recently?)
E.g. if one user downloads a container, and then for another user it is already in the cache, this gives the other user information about the first user.
I think that one of the reasons many people might not be aware of the VM is because -- in my experience -- Docker Desktop works almost identically to Docker on a real Linux system. I feel like Docker has done a fantastic job at making you feel like it's running natively (i.e. despite running in a VM you can mount volumes close to the same way, you can use the docker CLI from the host, etc.). Additionally, I don't think people realize/care that Linux containers rely heavily on features the Linux kernel provides (interestingly, and less well-known, Microsoft has done a lot of work to provide Windows containers[0], too).
I am curious, though, why in your use-case of Tailscale and Traefik knowing that Docker Desktop runs in a VM would impact anything from a functional standpoint? I.E. why would the VM have even been an important factor to the support teams you reached out to?
> This industry is just a bunch of overly paid amateurs
I think, perhaps a more compassionate view is that everyone is learning and growing and it's difficult to be an expert at literally everything you use in your stack. :)
[0] https://learn.microsoft.com/en-us/virtualization/windowscont...
When using Ubuntu/WSL2, there are not the same daemons running as on the same Ubuntu running on it's own. Tailscale expects one or two, I'd have to dig into my emails to find the specifics, something like no systemd under Ubuntu/WSL2 and Tailscale not checking, just failing. I seem to remember there was more than one daemon expected, which might be present on that other VM but either Tailscale or Traefik know to check or communicate with that other VM and their integration fails. Support's recommendation was to just use a no desktop gui server where everything just works.
Yeah, I get grumpy. I need to check myself better. I realize we're all trying our best.
Ah, makes sense. I have encountered some funky stuff with Docker+WSL, especially because I often prefer to use distros other than Ubuntu. It feels extra fragile/added complexity how Docker Desktop on Windows relies on WSL for Linux containers.
Thanks for indulging my curiosity!
I agree the initial setup is more difficult with a K8s local dev than with a Docker Compose local dev.
But the impedance mismatch makes using Docker Compose a bad idea when you deploy to a K8s cluster.
The secrets management is different. The way pods talk to each other is different. App-level configuration and updates are different. There is so much that diverges, that you end up with two independent systems. One for dev and one for prod. This is not good.
That's a pretty strong option!
at best, you can deploy that app to a Docker Swarm. Docker Swarm is not reasonably comparable to K8s.
there are tools (and colleagues ! :) ) to transform docker compose files to K8s manifests.
Docker Compose is still a moving project. Look at the version of the schema. It evolves.
Kubernetes is also a platform that is moving. And it is moving fast.
If Kompose was considered `done` by its authors in 2020, I don't think I would even bother trying it out. It probably wouldn't be able to parse recent Docker Compose schemas and it would probably output outdated Kubernetes manifests.
But, see, yesterday I was looking for a Time Boxing application and stumbled on that one: https://github.com/khrykin/StrategrDesktop/. I dismissed it because the last release was in 2020. But that is an example of an app that could be considered `done` and still useful 3 years later.
Hence why I partially agree with you. :)