Seems that people think Docker has gave in, but I am not that sure. If you can switch between Swarm/Kubernetes transparently, then why wouldn't you start with Swarm? (I'm talking about small companies who're just starting with containers.)

Why wouldn’t you start with Kubes? Less vendor lock in and a much bigger community.

Because K8s has quite some overhead, a steeper learning curve and is more difficult to set up and to maintain. I don't see this as a problem in bigger companies, but in companies with less than 10 IT people and no prior knowledge I'd take Swarm over K8s every day.

We have less than 10 people, 5 of them engineers. We are overbooked getting features out. We are using K8S in both dev and production, not Swarm.

A big part of that is because I have had experience running Docker (not Swarm), ECS, K8S, and building developer tooling (Vagrant), in addition to being a regular developer.

The flip side: I had to opportunity to try out these different tech in production and saw where the pain points are. The overhead of K8S exists to solve those pain points, though that is probably not that obvious to a small team without prior knowledge.

For example, I had set up a prod k8s by hand. I will never do that again. On the other hand, I know roughly what is going on when something breaks in our Google GKE cluster.

Hosh , could you share your learning from setting up a prod K8 HA cluster? Could be useful for me.

https://rocketeer.be/blog/2015/11/kubernetes-from-the-ground...

And although I never ran through Hightower's Kubernetes the Hard Way, it is like that. https://github.com/kelseyhightower/kubernetes-the-hard-way

After running through that as a kind of kata, it was easier to infer and troubleshoot things when things go wrong. The transfer-of-learning happens only if you run yourself through these exercises.

I can share some things at a higher level though:

Label selectors are your friend. Master them. They are used everywhere.

Stateless is still easier than stateful. Start with putting stateless workloads in production before ever trying stateful.

If you have the expertise to mix your stateful pods with your stateless pods, make sure you master StatefulSet and things like persistant volume claims.

If you fake stateful pods like I did in production, then Kubernetes does not know how to cleanly shut them down. Automated maintenance involving kubectl cordon and drain no longer function well. You end up having to hand migrate stateful pods from node to node.