I know this will cost me a lot of karma, but surely this is the point that you say, lets just use a hypervisor?

yes, I know its much easier to just use docker, but at the point that you have to write a new INIT, doesn't that strike you as time you used something thats not easy anymore?

Things like "make sure we don't loose syslog" make me shudder.

Yes supposedly there is an argument that docker is faster. However as a buildeng i've seen little evidence of that.

We use VMware with throwaway build disks. This means that a Dev can destroy a machine, once rebooted its back to a known default state.

the best part is that we have collectd on the vmware host, and the in the guest to collect metrics on CPU, mem and IO.

Because we have decent disks, and lots of memory, we don't hit any limitations. For heavy build we have NFS root'd servers with non persistant disks. much faster, and requires almost no thinking about.

Docker does different things than VM's, but they work together perfectly. I run my all my Docker hosts on a VMWare infrastructure, which gives me best of both worlds. Yes - for some things, VMWare can't be beaten - but that's the nice thing: they are not mutually exclusive. Both are very handy tools in my toolbox, and neither are a one-stop solution. Docker sure adds complexity - but also a huge amount of flexibility. Once you understand the power of it all, and how to think with Docker (which isn't that obvious) - it's pretty damn good. Not that there are no downsides or things that could be improved, but nothing is perfect.

And then:

1) The problem is hugely exaggerated. It only applies to badly written software. And make sure we don't lose syslog? Seriously? just mount /dev/log into your container (it's a unix socket after all) and you're done.

2) Arguably faster? Try booting up 15 instances of app X. With docker, that's achieved in less than a second. On VMWare this has a huge memory and CPU overhead, not to mention boot-up times.

3) Try versioning VM's. Good luck. I have 100% reproducible, versioned images generated and tested in our CI system, that I can easily deploy in a VM on my laptop where I will have 100% the same setup as it will in test/qa/prod. To deploy the new version, start new instance, point load balancer/proxy/whatever to the new instance (happens automagically btw). Everything goes well? Stop old instance. Something goes wrong? Point proxy to old instance, and kill the badly-behaving. If everything went well: zero downtime. If you detect something goes wrong, it's nearly instant to switch back to the previous version. Also, testing this is a lot easier than with VM's. Sure you could do that with VM's too - but the overhead is massive, and takes a lot more time.

My ideal setup would be a hypervisor infrastructure with a bunch of VM's running only Docker applications.

You should take a look at https://github.com/docker/machine and https://github.com/docker/swarm , they were designed with exactly that goal in mind!