Ask HN: What is the actual purpose of Docker?

313 points by someguy1233 10 years ago | 154 comments

I'm hearing about Docker every other day, but when I look into it, I don't understand the purpose of it.

I run many websites/applications that need isolation from each other on a single server, but I just use the pretty-standard OpenVZ containers to deal with that (yes I know I could use KVM servers instead, but I haven't ran into any issues with VZ so far).

What's the difference between Docker and normal virtualization technology (OpenVZ/KVM)? Are there any good examples of when and where to use Docker over something like OpenVZ?

tinco 10 years ago |

> What's the difference between Docker and normal virtualization technology (OpenVZ/KVM)? Are there any good examples of when and where to use Docker over something like OpenVZ?

Docker is exactly like OpenVZ. It became popular because they really emphasize their OpenVZ Application Templates feature, and made it much more user friendly.

So users of Docker, instead of following this guide: https://openvz.org/Application_Templates

They write a Dockerfile, which in a simple case might be:

    FROM nginx
    COPY index.html /usr/share/nginx/html

So no fuzzing with finding a VE somewhere, downloading it customizing it, and then installing stuff manually, stopping the container and tarring it, Docker does that all for you when you run `docker build`.

Then you can push your nice website container to the public registry, ssh to your machine and pull it from the registry. Of course you can have your own private registry (we do) so you can have proprietary docker containers that run your apps/sites.

From my perspective, the answer to your question would be: Always prefer Docker over OpenVZ, they are the same technology but Docker is easier to use.

But I've never really invested in OpenVZ so maybe there's some feature that Docker doesn't have.

segmondy 10 years ago | |

Docker and OpenVZ are not the same. Docker is single application focus. OpenVZ provides the entire OS in a container. OpenVZ has support for live migration.

tinco 10 years ago | | |

It could be that their communities have different philosophies about containers, but this is not a technical difference. I think Docker added support for live migration recently too (the Doom demo on that recent Docker con right?)

They are built around the same set of kernel features and as far as I can see offer exactly the same abstractions.

For example Phusion Baseimage is a Docker baseimage that's similar to an OpenVZ container in the sense that it emulates a full running Ubuntu environment. It has is uses but the Docker community rather sees containers that encapsulate a single application with no external processes.

klibertp 10 years ago | | |

> OpenVZ provides the entire OS in a container.

Docker does that, too. Actually, I'm running docker containers as a fast and easy replacement for VirtualBox VMs.

alinspired 10 years ago | | |

OpenVZ is closer to KVM (full virtualization). OpenVZ container is a lightweight VM which shares host kernel, has persistent FS and traditional OS you have to manage.

hueving 10 years ago |

It serves as an amazing excuse to re-invent the wheel at your own workplace. It's a hot technology, and if you're not using it, it's because you're inept. Rip all of the stable things out that everyone knew how to use and slap containers in there! If it's not working, it's because your not using enough containers.

No security patching story at your workplace? No problem, containers don't have one either! If someone has shipped a container that embedded a vulnerable library, you better hope you can get a hold of them for a rebuild or you have to pull apart the image yourself. It's the static linking of the 21st century!

jordigh 10 years ago | |

I want to downvote the first paragraph but upvote the second one.

Doesn't Docker also help cause problems like ssh private key reuse? I am sure that there are mitigations, but it's sad to have ways to prevent some activity that the software makes easy to do.

davexunit 10 years ago | | |

>I want to downvote the first paragraph but upvote the second one.

I had the very same feeling. Containers are very useful, but the Docker suite of tools just don't have a very good security story.

KaiserPro 10 years ago |

docker and openVZ aim to do the same thing.

docker is a glorified chroot and cgroup wrapper.

There is also a library of prebuilt docker images (think of it as a tar of a chroot) and a library of automated build instructions.

The library is the most compelling part of docker. everything else is basically a question of preference.

You will hear a lot about build once, deploy anywhere. whilst true in theory, your mileage will vary.

what docker is currently good for:

o micro-services that talk on a messaging queue

o supporting a dev environment

o build system hosts

However if you wish to assign ip addresses to each service, docker is not really mature enough for that. Yes its possible, but not very nice. You're better off looking at KVM or vmware.

There is also no easy hot migration. So there is no real solution for HA clustering of non-HA images. (once again possible, but not without lots of lifting, Vmware provides it with a couple of clicks.)

Basically docker is an attempt at creating a traditional unix mainframe system (not that this was the intention) A large lump of processors and storage that is controlled by a singular CPU scheduler.

However, true HA clustering isn't easy. Fleet et al force the application to deal with hardware failures, whereas Vmware and KVm handle it in the hypervisor.

grhmc 10 years ago |

For me, it is the ultimate in the idea in Continuous Delivery of "build once." I can be very confident that the docker image I build in the first stage of my pipeline will operate correctly in production. This is because that identical image was used for unit tests, to integration and functional testing, to the staging environment and finally production. There is no difference than configuration.

This is the core that Docker solves, and in such a way that developers can do most of the dependency wrangling for me. I don't even mind Java anymore because the CLASSPATHs can be figured out once, documented in the Dockerfile in a repeatable programatic fashion, and then ignored.

In my opinion the rest of it is gravy. Nice tasty gravy, but I don't care so much about the rest at the moment.

Edit: As danesparz points out, nobody has mentioned immutable architecture. This is what we do at Clarify.io. See also: https://news.ycombinator.com/item?id=9845255

shawnee_ 10 years ago |

Docker is a cute little tool that gives people who aren't that great at Linux the illusion that they know what they're doing. Throw in the use of some "Container" semantics and people become convinced it's that easy (and secure) to abstract away the containers from the kernel.

But it's not, at least in my experience; not to mention that as of now, anything running Docker in production (probably a bad idea) is wide open to the OpenSSL security flaw in versions of 1.0.1 and 1.0.2, despite the knowledge of this issue being out there for at least a few days.

Docker's currently "open" issue on github: https://github.com/docker/compose/issues/1601

Other references: https://mta.openssl.org/pipermail/openssl-announce/2015-July... http://blog.valbonne-consulting.com/2015/04/14/as-a-goat-im-...

alextgordon 10 years ago |

1. Stateless servers. Put your code and configuration in git repos, then mount them as volumes in your docker container. The absolute star feature of docker is being able to mount a file from the host to the container.

You can tear down the host server, then recreate it with not much more than a `git clone` and `docker run`.

2. Precise test environment. I can mirror my entire production environment onto my laptop. No internet connection required! You can be on a train, on a plane, on the beach, in a log cabin in the woods, and have a complete testing environment available.

Docker is not a security technology. You still need to run each service on a separate host kernel, if you want them to be properly isolated.

davexunit 10 years ago | |

>The absolute star feature of docker is being able to mount a file from the host to the container.

This is a simple bind-mount and isn't special at all.

    mount("/foo", "/container/foo", "none", MS_BIND);

Also, virtual machines have had things like 9p that allow the same thing.

alextgordon 10 years ago | | |

I don't think there is enough RAM in my laptop to run five VMs simultaneously :)

danesparza 10 years ago |

I'm stunned that nobody has brought up the idea of 'immutable architecture' -- the idea that you create an image and deploy it, and then there is no change of state after it's deployed. If you want a change to that environment, you create a new image and deploy that instead.

Docker gives you the ability to version your architecture and 'roll back' to a previous version of a container.

stephengillie 10 years ago | |

In the full-OS VM (KVM, VMWare, etc) this is known as disk snapshotting. Another way to look at it is putting an RDBM in full-recovery mode, so the database itself remains the same and replaying logs is required to get the data's true state.

You shut down a VM and instruct the hypervisor system to take a "snapshot" which locks the original VHD file and creates a new one. When writes happen, they're performed on the new VHD, and reads have to use both the main and the snapshot VHD. And you can create a chain of snapshots, each pointing to the previous snapshot, for versioning. Or you can have several VM snapshots use the same master VHD, like for CI or data deduplication.

To roll back, it's usually as simple as shutting down the VM and removing the snapshot file.

hueving 10 years ago | |

Nobody is mentioning it because VM's already did this for more than a decade.

hosh 10 years ago | | |

This isn't true.

The way VMs handle this doesn't carry the same semantics as the way you can with Docker. There's a finer-grain composability with Docker that is much more awkward with VMs.

Docker may not be as great as a virtualization tool as VMs -- security concerns, complexity, etc. -- but it is a much better package management tool.

crdoconnor 10 years ago | |

This means that you have to roll out a (potentially huge) new blob each time you want to make even small config changes.

You get most of the benefits of immutable builds anyhow by having scripts which can reliably set up servers from scratch on the fly.

IanCal 10 years ago | | |

This is where the layers come in useful, pulling a small change should only require pulling a small new layer.

marcosdumay 10 years ago | |

My shell glue and apt-magic I wrote for my personal servers does that too... And people have been doing that for decades, just not as often, because servers used to be expensive.

zwischenzug 10 years ago |

Some key points:

- Docker is nothing new - it's a packaging of pre-existing technologies (cgroups, namespaces, AUFS) into a single place

- Docker has traction, ecosystem, community and support from big vendors

- Docker is _very_ fast and lightweight compared to VMs in terms of provisioning, memory usage, cpu usage and disk space

- Docker abstracts applications, not machines, which is good enough for many purposes

Some of these make a big difference in some contexts. I went to a talk where someone argued that Docker was 'just a packaging tool'. A sound argument, but packaging is a big deal!

Another common saw is "I can do this with a VM". Well, yes you can, but try spinning up 100 vms in a minute and see how your MacBook Air performs.

3pt14159 10 years ago | |

What is important to note is that the ideal use case for Docker is not actually speed - it's about quickly responding to increased load. If you need to vary the number of, say, web servers from 5 to 100 over the day, then Docker is good because you can start them up very quickly.

The down side about docker is that it takes longer to set up your docker in the first place, it is harder to keep secure, and it runs slower than a traditionally deployed application, but compared to VM deployed applications the performance is usually better.

arenaninja 10 years ago | |

Docker is indeed fast and lightweight. It's amazing how much CPU power is freed up from not running a full on VM in VirtualBox. That said, I'm wary of running it in production

yellowapple 10 years ago | | |

In my experience, the performance gains are much less significant v. a virtual machine running on a proper bare-metal hypervisor (like Xen).

zwischenzug 10 years ago | | |

Why? Security?

hosh 10 years ago | |

Yes, packaging is a big deal!

akshaykarle 10 years ago |

Docker is mainly an app packaging mechanism of sorts. Just like you would build a jars, wars or rpms, etc. you create docker images for your applications. The advantage you get is that you can package all your dependencies in the container thereby making your application independent and using the tools provided by docker in combination with swarm, compose, etc. it makes deployment of your apps and scaling easier.

OpenVZ, LXC, solaris zones and bsd jails on the other hand or mainly run complete OS and the focus is quite different from packaging your applications and deployments.

You can also have a look at this blog which explains the differences more in detail: http://blog.risingstack.com/operating-system-containers-vs-a...

jacques_chester 10 years ago |

Docker uses the same kernel primitives as other container systems. But it tied together cgroups, namespaces and stackable filesystems into a simple cohesive model.

Add in image registries and a decent CLI and the developer ergonomics are outstanding.

Technologies only attract buzz when they're accessible to mainstream developers on a mainstream platform. The web didn't matter until it was on Windows. Virtualization was irrelevant until it reached x86, containerization was irrelevant until it reached Linux.

Disclaimer: I work for a company, Pivotal, which has a more-than-passing interest in containers. I did a presentation on the history which you might find interesting: http://livestre.am/54NLn

bosse 10 years ago | |

As an ops guy, I would also mention the benefits of the Dockerfile and docker-compose.yml, which could be clear sources for information to how the system is built, and which in most circumstances would build the same system in dev as in prod. By changing a docker tag in the configuration management, I can roll out a new version quite conveniently to staging and eventually to production.

The potential minimalism of a container is also an important concept to mention, with fast startup-times and less services that could potentially be vulnerable.

patsplat 10 years ago | | |

Agreed.

Application runtime dependencies are a common source of communication breakdowns between development and infrastructure teams. Making the application container a maintained build file on the project improves this communication.

docker provides:

* a standard format for building container images (the Dockerfile)

* a series of optimizations for working with images and containers (immutable architecture etc).

* a community of pre-built images

* a marketplace of hosting providers

All at the cost of linux only, which is ok for many shops.

sudioStudio64 10 years ago |

I think the main thing is to provide an abstraction for applications so that they aren't tightly coupled to the operating system of the server that's hosting them. That's a big deal.

Some people have mentioned security...patching in particular. Containers won't help if you don't have patching down. At the very least it lets you patch in the lab and easily promote the entire application into production.

I think that the security arguments are a canard. By making it easier and faster to deploy you should be able to patch app dependencies as well. I, for one, would automate the download and install of the latest version of all libs in a container as part of the app build process. Hell, build them all from source.

IT departments need to be able to easily move applications around instead of the crazy build docs that have been thrown over the wall for years.

jtwebman 10 years ago |

It's a tool to make over engineering every project even easier! All joking aside it is a good tool for some teams to make sure the same exact code is running in production that was tested. I don't think it is for everyone and can make things much more complicated than they need to be. I also don't think everything needs to be in a docker.

hmans 10 years ago |

Docker is the industry-accepted standard to run web applications as root.

davexunit 10 years ago | |

It's unfortunate that Docker still doesn't use user namespaces.

justincormack 10 years ago | | |

No, they are new and have had many security issues. Just run your containers not as root, you can use capabilities if you like.

csardi 10 years ago |

I like this presentation, as it shows what Docker really is, and also how to use Docker without Docker: https://chimeracoder.github.io/docker-without-docker/#1

mariocesar 10 years ago |

The most common pro is "Build once deploy everywhere" even is possible, I always feel pushing a 500 MB tar image to the production servers is more an annoyance than being helpful; Yes, You can setup your own registry but maintaining the service, securing, adding user permissions and maybe use a proper backend like S3 is an extra annoying layer and another component that could fail.

If the docker tool will have something like `docker serve` and start his own local registry will be more than great.

For this case when I switch to Go was a great solution, building the binary is everything you need.

About docker being helpful for development, definitively yes, I switch to postgres, elasticsearch and redis containers instead of installing them on my computer, is easy to flush and restart and having different versions of services is also more manageable

spaceisballer 10 years ago |

I know you have some other questions that I am not qualified to answer, but I recalled seeing a similar question asked not that long ago.

https://news.ycombinator.com/item?id=9805056

dschiptsov 10 years ago |

To create a buzzword to attract investors money. It is professional brand management at work.

hosh 10 years ago |

You're coming at this from the wrong direction, namely virtualization.

What differentiates Docker is not virtualization, so much as package management. Docker is a package management tool that happens to allow you to execute the content of the package with some sort of isolation.

Further, when you look at it from that angle, you start seeing the flaws with it, as well as it's potential. It's no accident that Rocket and the Open Container Project are arising to standardize the container format. Other, less-well-known efforts include being able to distribute the container format themselves in a p2p distribution system, such as IPFS.

Kiro 10 years ago |

For me I don't understand the purpose at all. I have a few node.js and PHP services. Why do I need isolation and have them in containers? If I want an identical environment when developing I can use Vagrant.

SEJeff 10 years ago | |

FYI: Vagrant can use docker, rendering your argument invalid :)

http://docs.vagrantup.com/v2/provisioning/docker.html

Docker is about running isolated environments in reproducible ways. I get a container working just so on my desktop, ship it to an internal registry, where it gets pulled to run on dev and qa. It works identically to how it works on my desktop, then I ship it to production. One image that works the same on all environments. That is what docker was for, developer productivity.

pjc50 10 years ago |

The description on HN the other day of Docker as a souped-up static linking system is the most interesting one.

tobbyb 10 years ago |

OpenVZ or LXC give you OS containers like KVM or VMWare gives your Virtual machines. Unlike OpenVZ, LXC does not need a custom kernel, and is supported in the mainline Linux kernel paving the way for widespread adoption.

Docker took the LXC OS container template as a base, modified the container OS init to run a single app, builds the OS file system with layers of aufs, overlayfs, and disables storage persistence. And this is the app container.

This is an opinionated use case of containers that adds significant complexity, more a way to deploy app instances in a PAAS centric scenario.

A lot of confusion around containers is because of the absence of informed discussion on the merits or demerits of this approach and the understanding that you have easy to use OS containers like LXC that are perfectly usable by end users like VMs are, and then app containers that are doing a few more things on top of this.

You don't need to adopt Docker to get the benefits of containers, you adopt Docker to get the benefits of docker and often this distinction is not made.

A lot of users whose first introduction to containers is Docker tend to conflate Docker to containers, and thanks to some 'inaccurate' messaging from the Docker ecosystem think LXC is 'low level' or 'difficult' to use, Why would anyone try LXC if they think it's low level or difficult to use? But those who do will be pleasantly surprised how simple and straightforward it is.

For those who want to understand containers, without too much fuss, we have tried to provide a short overview in a single page in the link below.

https://www.flockport.com/containers-minus-the-hype

Disclosure - I run flockport.com that provides an app store based on LXC containers and tons of tutorials and guides on containers, that can hopefully promote more informed discussion.

theknarf 10 years ago |

Docker is a way to create immutable infrastructure, which is a key component to a) have software working the same in test and prod. (hint DevOps.) and b) creating servers which can scale both vertically and horizontally.

I think thats the best way I can summarise what Docker _is_.

mbrock 10 years ago |

I don't know much about virtualization technology, but Docker is nice for me because it's an accessible, well-known, and rather easy way to make applications easy and straightforward to run.

Where I've worked in the past, setting up a new development or production environment has been difficult and relied on half-documented steps, semi-maintained shell scripts, and so on. With a simple setup of a Dockerfile and a Makefile, projects can be booted by installing one program (Docker) and running "make".

You could do that with other tools as well, but Docker, and even moreso the emerging "standards" for container specification, seems like an excellent starting point.

bfirsh 10 years ago |

This explains the difference between Docker and normal virtualization technology: https://www.docker.com/whatisdocker

corradio 10 years ago |

Might be interesting for you: https://medium.com/using-artificial-intelligence-to-make-tec...

johnminter 10 years ago |

I think one useful purpose was described by Prof. Mine Cetinkaya-Rundel of Duke at the recent UseR conference. She teaches an introductory statistics class for non-majors. Docker lets her spin up individual virtual machines for each student with all the packages they need for the class without all the sys-admin headaches of getting all the software on everybody's systems. You can see her slides and evaluation of the alternatives here:

https://github.com/mine-cetinkaya-rundel/useR-2015/blob/mast...

lgunsch 10 years ago |

Simply put, Docker is operating system virtualization:

https://en.wikipedia.org/wiki/Operating-system-level_virtual...

Edit: formatting.

somberi 10 years ago |

A meta critique after reading 139 comments: I too had the same question as the parent and from the ensuing conversations, I assume that either Docker is so thin-layered (not in a bad way) that it is open to so many interpretations or it is so thin-layered (in a trivial way), that one does not need to get all worked up adopting it if one is comfortable in using other VM options out there (like OpenVZ for example).

theneb 10 years ago |

I find docker is quite good for integration tests where you need to test against a third party bit of software. Lots of images exist in the hub for this.

justincormack 10 years ago |

OpenVZ is not upstream in the kernel; the container stuff that got merged is what Docker uses. Docker has much wider adoption than OpenVZ does now.

jdoss 10 years ago | |

> Docker has much wider adoption than OpenVZ does now.

I don't think your statement is true at this point in time. OpenVZ is used by a ton of companies in the hosting industry and by large companies such as Groupon and smaller ones like TravisCI [1]. I would't make a statement that that Docker has a wider adoption than OpenVZ at this point in time. Maybe in five years, yes it may have a wider adoption than OpenVZ. OpenVZ and commercial VZ have been doing full OS containers since the early 2000s and it has the production track record to do very well in many server applications. I wouldn't hesitate to use it over Docker in production for my future projects.

[1]: http://changelog.travis-ci.com/post/45177235333/builds-now-r...

justincormack 10 years ago | | |

Travis moved to Docker after that [1]. And the "hosting industry" is not the thing it used to be since cloud.

[1] http://blog.travis-ci.com/2014-12-17-faster-builds-with-cont...

SEJeff 10 years ago | | |

And Docker is massively used at Groupon, so your argument isn't valid.

https://engineering.groupon.com/2014/misc/dotci-and-docker/ http://www.meetup.com/Docker-Chicago/events/220936626/

Source: I work at 600 W Chicago, the Groupon World HQ, where they frequently host Docker meetups on the 6th floor.

xaduha 10 years ago |

Someone (Darren Shepherd?) compared Docker to Ajax. It's not a technological breakthrough, it's another kind of breakthrough.

I think it was here [1], but deleted now.

[1] http://ibuildthecloud.tumblr.com/post/63895248725/docker-is-...

tfn 10 years ago |

I went ahead and blogged an answer here: http://blog.tfnico.com/2015/07/the-sweet-spot-of-docker.html

TL;DR: It's better for deploying applications and running them than using home-made scripts.

atsaloli 10 years ago |

See http://stackoverflow.com/questions/29304951/difference-betwe...

programminggeek 10 years ago |

It exists to create jobs in devops.

kolyshkin 10 years ago |

[Disclaimer: I am the guy who was running OpenVZ since the very beginning, and if you hate OpenVZ name/logo, I am the one to blame. Also, take everything I say with a grain of salt -- although I know, use, like and develop for Docker, my expertise is mostly within OpenVZ domain, and my point of view is skewed towards OpenVZ]

Technologically, both OpenVZ and Docker are similar, i.e. they are containers -- isolated userspace instances, relying on Linux Kernel features such as namespaces. [Shameless plug: most of namespaces functionality is there because of OpenVZ engineers work on upstreaming]. Both Docker and OpenVZ has tools to set up and run containers. This is there the similarities end.

The differences are:

1 system containers vs application containers

OpenVZ containers are very much like VMs, except for the fact they are not VMs but containers, i.e. all containers on a host are running on top of one single kernel. Each OpenVZ container has everything (init, sshd, syslogd etc.) except the kernel (which is shared).

Docker containers are application containers, meaning Docker only runs a single app inside (i.e. a web server, a SQL server etc).

2 Custom kernel vs vanilla kernel

OpenVZ currently comes with its own kernel. 10 years ago there were very few container features in the upstream kernel, so OpenVZ has to provide their own kernel, patched for containers support. That support includes namespaces, resource management mechanisms (CPU scheduler, I/O scheduler, User Beancounters, two-level disk quota etc), virtualization of /proc and /sys, and live migration. Over ten years of work of OpenVZ kernel devs and other interesting parties (such as Google and IBM) a lot of this functionality is now available in the upstream Linux kernel. That opened a way for other container orchestration tools to exist -- including Docker, LXC, LXD, CoreOS etc. While there are many small things missing, the last big thing -- checkpointing and live migration -- was also recently implemented in upstream, see CRIU project (a subproject of OpenVZ, so another shameless plug -- it is OpenVZ who brought live migration to Docker). Still, OpenVZ comes with its own custom kernel, partly due to retain backward compatibility, partly due to some features still missing from the upstream kernel. Nowadays that kernel is optional but still highly recommended.

Docker, on the other side, runs on top of a recent upstream kernel, i.e. it does not need a custom kernel.

3 Scope

Docker has a broader scope than that of OpenVZ. OpenVZ just provides you with a way to run secure, isolated containers, manage those, tinker with resources, live migrate, snapshot, etc. But most of OpenVZ stuff is in the kernel.

Docker has some other things in store, such as Docker Hub -- a global repository of Docker images, Docker Swarm -- a clustering mechanism to work with a pool of Docker servers, etc.

4 Commercial stuff

OpenVZ is a base for commercial solution called Virtuozzo, which is not available for free but adds some more features, such as cluster filesystem for containers, rebootless kernel upgrades, more/better tools, better containers density etc. With Docker there's no such thing. I am not saying it's good or bad, just stating the difference.

This is probably it. Now, it's not that OpenVZ and Docker are opposed to each other, in fact we work together on a few things:

1. OpenVZ developers are authors of CRIU, P.Haul, and CRIU integration code in Docker's libcontainer. This is the software that enables checkpoint/restore support for Docker.

2. Docker containers can run inside OpenVZ containers (https://openvz.org/Docker_inside_CT)

3. OpenVZ devs are authors of libct, a C library to manage containers, a proposed replacement or addition to Docker's libcontainer. When using libct, you can use enhanced OpenVZ kernel for Docker containers.

There's more to come, stay tuned.

droidztix 10 years ago |

reading while eating popcorn ( ͡° ͜ʖ ͡°)