We will be happy if Docker talks about Swarm becoming a management UX for K8s - but we need visibility. These are production orchestration systems. The migration path is not easy.
And seeing what Docker Co is doing with Cloud, it is not very comforting to trust that they will do the right thing with Swarm.
We followed Swarm from the beginning, but after a few releases at v0.4 it was clear not to ever use Swarm, and that it mostly was the Docker PR machine that made it sound nice, and not the actual features.
Maybe it got better later on, but the first several Swarm announcements seemed really off-putting to me.
We ended up on Mesos/Marathon, not that that has a bright future either, but it was at least capable of restarting containers from the beginning..
Just migrate to Kubernetes. It has won.
I've been taking a second look at Mesos recently and found that I didn't really grok it the first time I looked at it. In any case, I think your assessment is correct ("Just migrate to Kubernetes" - https://trends.google.com/trends/explore?date=today%205-y&q=...).
Can you share some 1st person opinions on Mesos, and where K8s is a step forward, backward or aside?
TIA
You can get a swarm cluster running in less than 10 minutes on your local laptop after "apt-get install docker-ce". To run k8s, you will need to first muck about ingress and overlays and everything else.
I know its because of "flexibility" - its like Sinatra vs Rails. They are both great in their spaces.
Stop spreading FUD please, it is not good for anyone.
Which is a pity because I really liked Swarm for its simplicity.
Side note: I am also concerned about Docker in general. CE/EE split, services shutting down, bugs seemingly not being fixed - I cannot point out a precise aspect, but I am concerned.
In our tests about a year ago, swarm started showing serious networking and cluster synchronization problems with cluster sizes over 30 nodes (physical servers), on a fast, reliable LAN.
I've heard similar stories from another big Docker customer- Docker support promised them that improving performance of Swarm and fixing scaling issues are the focus of "the next version", but they never came. This company is now moving to k8s.
Public statement on their blog after the K8s announcement in EU:
"But it’s equally important for us to note that Swarm orchestration is not going away. Swarm forms an integral cluster management component of the Docker EE platform; in addition, Swarm will operate side-by-side with Kubernetes in a Docker EE cluster, allowing customers to select, based on their needs, the most suitable orchestration tool at application deployment time."
https://blog.docker.com/2017/11/swarm-orchestration-in-docke...
There's still plenty of PR's and activity in the SwarmKit and Libnetwork repos:
https://github.com/docker/swarmkit/pulse/monthly https://github.com/docker/libnetwork/pulse/monthly
Then, they cancelled it, no more patches, no mention of it anywhere unless you know where to look.
Then they created Swarm Mode, and add the concept of "services" which sucks compared to regular run because it lacked so many options the run command had, it took more than 6 months to implement most of them.
To be fair docker cloud was never great and hopefully it doesn't have many big customers...
But the precedence of shutting down a paid service with 2 months notice is not nice. What would happen if this was docker hub? Panic!
People. People are up in arms.
Anyway, yeah, that's insane. Even Google, who constantly shuts stuff down, usually does so with way more heads up. For comparison, Google Reader, a completely free service, shut down with 3.5 months advance notice. Google Wave got almost 6 months notice.
It still hurts.
Docker cloud is no loss (with apologies to those who are using it in production) and will hopefully free up your people to work on other important stuff.
Otherwise if we can continue to use the compose configuration api and the docker deploy/service api with k8s under the hood then I guess that's a reasonable compromise.
I imagine the service wasn't really making money, and to add GDPR compliance on top, could have been the last straw (or a contributing factor)
On a meta thought, I wonder what potentially caused this move. It is/was a pretty decent service.
They should have announced it earlier and should have given more time to paying customers. I am glad I migrated very early.
I played around with it a long while ago and really liked it, but felt like more moving parts (and particularly on MySQL, which wasn't in our stack) wasn't something I was too keen on. Having someone manage this for us could be worth paying for though.
And it always lost its ip sec connections and only a reboot helped.
I liked it though...
I think Microsoft still employs thousands of great engineers and have been early embracers of containerization among the large companies out there and because Satya was a large part of growing Azure into what it is (IMHO a pretty solid set of services) it could make a lot of sense
Tutum actually worked and I really liked it. Now I plan to use Rancher on top of kubernetes for my docker hosting
Docker Cloud was getting worse and worse. Tutum (they acquired, rebranded and killed it) was great. But docker team just destroyed it.
That is a shame. Tutum was great since it scaled up and down very well. Nobody thinks about scaling down, but this is important in many ways.
Right now IMO: - Docker swarm - scales up and down ok. Had too many bugs with even on stable. - Rancher - fine, very good for medium deployments. - k8 - winner for larger scale but scales down badly.
This is really sad.
I've been working on a Docker Cloud alternative for awhile now. I'm aiming for something that kind of balances the convenience of Heroku with the Docker experience.
It's still in beta but if anyone wants to check it out it's at: https://codemason.io/
Kubernetes “won” so now the competition has shifted from who can innovate to who can execute and operate.
Well, "won" in quotes. I am not a k8s fanboy or anything, I simply observe that all the major cloud providers are offering managed k8s services that have superseded their own proprietary container-type offerings. For better or worse that's where the momentum is. If you wanted to containerize your stack right now, k8s then pick one of the big 3, seems like a safe bet.
There's a reason why people and businesses hate vendor lock ins. There's a reason why we do not want AWS to reign supreme for long.
Isn't docker hub relatively easy to replace, in comparison?
Otherwise, is this a wakeup call perhaps?
Traditional package managers have two distinct concepts: a repository and packages within those repositories.
For example, if ubuntu took down their apt repo server, I could run my own with all the same packages and change a single sources.list entry and all my servers, ansible roles installing packages, etc, would operate the same.
This is possible because the package name+version is an identifier everything else uses and the only thing that cares about the repository is apt itself; all other tooling doesn't need to know about the repository the package is sourced from.
Docker conflates those two things. Each client doesn't just send a package name, it sends a url + package name + version (e.g. foo.registryurl.com/image:version). Because every single client has the detail of "foo.registryurl.com" baked in, it's difficult to change that. I can't change a single "repository-mapping" file that the docker daemon reads to quickly update it.
Instead, I have to update every single client.
The idea of decoupling those is not new. In 2014 it was proposed [1], and various implementations that would help make it easier to migrate off the default registry have been proposed and rejected [2]
This doesn't even get into the lack of tooling for chasing down the transitive dependencies building my images has on various registries with each FROM.
[1]: https://github.com/moby/moby/issues/8329
[2]: https://github.com/moby/moby/pull/5821#issuecomment-49492924
Migrating off docker cloud will be a pain, but the service was already a pain to use, so maybe it's about time anyways.
But imagine being given 2 months to migrate off docker hub for image storage. Panic would ensue :)
edit: typos
The whole point of containers is that they are ephemeral and can be booted up quickly anywhere because you statically link the whole fucking OS?
Docker the company have few options to monetize on Docker the software when it becomes commoditized. They seeminly chose the Enterprise way, which consists of pretty orchestration tools and integrations with Active Directory. (A perfectly valid option, which worked out well for VMware.) That's a dead end now that Kubernetes won container orchestration. It will be interesting to see where they will go next.
It's not a completely fair comparison since we also were able to offload persistent storage to Google Cloud, which is one of the harder problems IMO.
I think Mesos has improved since then, but it always felt like they were a bit behind.
In general Kubernetes feels like it is designed by people with relevant experience. Especially compared to our earlier experiments with Docker Compose files. People are praising their simplicity, but they left us solving a lot of hard problems that Kubernetes solves for us better than we could have done.
Rancher (and Rancher OS) seem to be doing fine and getting updates constantly.
We're eyeing Rancher 2.x and the Kubernetes integration but it gives me confidence seeing Rancher 1.x getting updated while they are so focused in 2.x and K8s.
Plus it's a self hosted version so no such fear of it disappearing within 2 months. :)
7.2 Deprecation Policy. Google will announce if it intends to discontinue or make backwards incompatible changes to the Services specified at the URL in the next sentence. Google will use commercially reasonable efforts to continue to operate those Services versions and features identified at https://cloud.google.com/terms/deprecation without these changes for at least one year after that announcement, unless (as Google determines in its reasonable good faith judgment):
(i) required by law or third party relationship (including if there is a change in applicable law or relationship), or
(ii) doing so could create a security risk or substantial economic or material technical burden.
The above policy is the "Deprecation Policy."
Disclaimer I work for Google in Cloud.
Just run a custom GitLab instance, and you get all that for free.
For my open source projects on git.kuschku.de I also have a gitlab container registry on k8r.eu, and it’s been amazing to work with.
I have only user rss via gwene over nntp, if that went away, i'd miss it bunches.
Integrating Nomad with Vault and Consul is super easy and allows you to provide secrets, configuration and service discovery to the application with the right layer of abstraction, the application should not be aware of the scheduler it is running on. Cloud auto join allows super easy cluster config. Job files are declarative.
Yes, Nomad does not have all the features of Kubernetes, but we take a different approach believing in workflows and the unix phillosophy of a single tool for a single job. A fairer comaprison would be to comapare the HashiCorp suite of OSS tools to K8s, Nomad, Vault, Consul, Terraform, this gives you capabilities to manage your workload both legacy and modern.
Looking at doing Vault in HA leads people to look at Consul, which leads to Nomad. (Consul uses a consensus protocol for service discovery and I think that will be interesting for the next generation).
Last year, K8S had already captured the center of gravity, and it took a while for the rest of the dev community to catch up.
I think this year ia a lot of shuffling as the survivors settles into orbit around K8S. There is a lot of interesting innovations up the stack once orchestration is de facto standardized.
K8S still hasn't solved the stateful workloads, though it is introducing a lot of primitives to support those: controller hooks, third part resources, on top of which Operators can function.
I think we will see a lot more innovations as people create Operators. That can include anything from stateful loads for specific distributed stateful loads, to things like intrusion detection, ML-driven autoscaling, and so forth.
$ brew cask install minikube
$ minikube startDocker Swarm runs exactly the same way with exactly the same components and with the same ease on laptop as well as the cloud.
TL;DR - you cant run minikube in production.
Nit: $ minikube get-k8s-versions The following Kubernetes versions are available when using the localkube bootstrapper: - v1.9.4 - v1.9.0 - v1.8.0 - v1.7.5 - v1.7.4 - v1.7.3 - v1.7.2 - v1.7.0 - v1.7.0-rc.1 - v1.7.0-alpha.2 - v1.6.4 - v1.6.3 - v1.6.0 - v1.6.0-rc.1 - v1.6.0-beta.4 - v1.6.0-beta.3 - v1.6.0-beta.2 - v1.6.0-alpha.1 - v1.6.0-alpha.0 - v1.5.3 - v1.5.2 - v1.5.1 - v1.4.5 - v1.4.3 - v1.4.2 - v1.4.1 - v1.4.0 - v1.3.7 - v1.3.6 - v1.3.5 - v1.3.4 - v1.3.3 - v1.3.0
With hosted Kubernetes as a service (GKE, AKS and soon EKS), there is little reason to roll your own cluster.
You can run Kubernetes on your Linux device machine as-is.
(I think this is the correct order, not 100% sure of course)
1) google borg (maybe omega) [1]
2) amazon ec2
3) whatever microsoft is using
(large gap)
4) all the rest of the world combined, a small portion of which is k8s
[1] https://www.quora.com/Does-Google-use-the-Open-Source-Kubern...
(one might even say [1] seems to imply it'll never happen, or at least take a very long time. Also if you read the papers it becomes very clear that "Google Borg" includes a lot of things these days at many levels, from custom ASICs, device firmware (as in standard device, google borg firmware), BIOS firmware, entirely custom sub-kernel code, custom kernels, custom userspace (ie. Google-specific libc that's not optional), ... all of these will turn out to have dependencies on eachother that have to be redone for k8s, could take a while to migrate over)
(although I have not read any papers on it (I'd love some though), I'd bet amazon is in a similar boat, and of course Microsoft is Microsoft)
See https://azure.microsoft.com/en-us/services/container-service... and https://aws.amazon.com/fargate/
This is very interesting. Could you talk more on this ? There is definitely space for an "opinionated k8s distro with batteries included". I have wished for Swarm to become this....
To combat this, every single image we use from hub.docker.com is "proxied" into our registry with a one-line Dockerfile:
FROM image:version
I’ve been trying to insulate myself from docker too and the FROM proxy strategy seems to break the least stuff. Have you hit any pain points?
We use orphan branches, one per image, although other strategies are possible (like using the commit diff and directory name).
Proxy images are versioned using branch names (e.g postgres vs postgres-9.6), images are pushed to gitlab.example.com/docker/library/postgres, and using version detection we generate docker image tags (e.g a 'postgres' branch will create postgres:latest, plus extracting version from postgres --version also pushes postgres:10 and postgres:10.1 images.
See this .gitlab-ci.yml[0]. Yes, there is one per branch. This can be generalised further (especially with Gitlab's new import system for .gitlab-ci.yml) but works well enough in practice, it's very low maintenance, and updates are a mere commit+push away.
In fact we use this not just for proxying images but for all "generalised", "utility", or "dependency" images that are not the result of a given full-blown app project in its own repo (those have their own CI/CD process in their respective repo)
All you need is an option to set the default registry. Probably it's already there, didn't google.
Compared to Kubernetes, Swarm is a breeze to setup, deploy and manage. The manifest files are the same Docker Compose files we're used to, just expanded to cover the new stack concepts. It has support for remote storage mounting, advanced networking configuration, various interesting volume and network plugins[0], and is generally a pain-free experience to use (from my admittedly short time with both).
Kubernetes is a fine product. It's just a shame Swarm doesn't seem to have the same traction.
Can someone share their Swarm experience in production, possibly compared to k8s?
I would love it (and pay for it) if Swarm says that it is a opinionated distro (ingress/overlay) + management layer on top of k8s.
The silence is deafening and not nice.
kubernetes got released more than a year earlier. And it wasn't until 2016 that people actually started to take docker-swarm seriously. Thats a two year headstart...
docker-swarm would've needed several really impactful feature to offset that. It didn't really. The only real upsite is how easy it is to install and maintain. But that's hardly important with pretty cheap hosted solutions around (at least cheap in comparison to hiring several SRE to maintain the cluster).
But thats just my opinion as a user.
Indeed - something like AKS can get you up and running very quickly, whereas as recently as 6 months ago unless you had a k8s wizard-guru on staff, there was no point in even trying to go it alone, esp. not into Production.
It also suggests that the skill set of the on-prem k8s expert will decline in demand over time, there will be less demand for people who can set it up from scratch on bare metal. We shall see!
Also, there are very large Swarm installations in production at large companies, so I'd be surprised if Docker cancelled the product (which is their flagship).