Ask HN: Teams migrated from Nomad to Kubernetes,what pushed you to migrate?

13 points by _shadi 3 years ago | 4 comments

we are trying to pick a self-hosted container orchestration engine to use, after evaluating the pros and cons of both we decided Nomad would be the better option for us.

I used both in the past, and for self-hosted solution I also prefer the simplicity of Nomad, but I realize that Kubernetes has much more functionality, so I'm wondering if there is something that we will realize that we are missing once we start using Nomad for production workloads.

So my question is targeted at people who used Nomad in production, and later found themselves migrating to Kubernetes, what is your story? and what functionality did you miss the most that pushed you to move to Kubernetes. Blogs and articles are also welcome.

twunde 3 years ago |

I've worked on a fargate + lambda migration to Nomad. At this point I definitely would have preferred to have gone with k8s. Nomad is still relatively complex, to the point where K8s is only somewhat more complex but made up with a bigger community and more tooling and features.

Features I miss: - ephemeral containers - dns integration with cloud providers - Up to date tutorials/examples, especially complex examples - Better out of the box validation/linting

sofixa 3 years ago | |

Disclaimer: I work at Hashicorp, not on Nomad (but I've been a fan long before joining) opinions my own, etc.

> dns integration with cloud providers

You have that, with an intermediary in the form of Consul Terraform Sync (runs Terraform based on events in Consul like new service or node - so you can have a terraform module doing your DNS integration, another opening firewalls, etc. all run after a service has been added/updated).

> Better out of the box validation/linting

How so? Both do fine client side validation/linting out of the box.

Ephemeral containers would be really cool to have.

proxysna 3 years ago |

Migrated from k8s to nomad. Did not miss anything, but apparently nomad can't do VM live migrations. It is not a part of our workflow when working with VM's, but still might be an issue for some.

verdverm 3 years ago |

Not a migration, but the team chose k8s because it had better docs and they were able to get much further with k8s than the others in the same amount of time. (they compared 4 options)