What does HackerNews think of linuxkit?
A toolkit for building secure, portable and lean operating systems for containers
[1]: https://github.com/linuxkit/linuxkit/blob/master/ADOPTERS.md
- some run only one language
- some require recompilation
- some essentially swap out libraries, others do something closer to dropping your already mostly static binary in a minimal disk image
- some build pid1 processes, others VMs images
Anyway, here are some additional entries in the space:
- https://ssrg-vt.github.io/hermitux/
- https://github.com/linuxkit/linuxkit (more embedded/minimal VM than unikernel)
- https://nabla-containers.github.io/ (runs on Solo5)
I am going through using Linuxkit to build AMIs for cloud providers now from containers. I wouldn’t necessarily class linuxkit as a unikernel project because it doesn’t have the hallmark blurring of user and kernel space or kernel-as-a-library but you can customize the kernel so it’s an adjacent idea, and I think it’s the one most likely to be in actual use at non-hyperscalers.
[EDIT] Added OSv since it's on one of the linked lists but is a pretty large active player in the field.
It's clear that bigger players, such as Red Hat[2] are interested in the topic of unikernels, and that cloud providers are preparing for this future too [3].
[0]: https://github.com/linuxkit/linuxkit
[1]: https://github.com/hckuo/Lupine-Linux
[1] https://developer.apple.com/documentation/hypervisor [2] https://github.com/machyve/xhyve [3] https://github.com/linuxkit/linuxkit
I aim to build a platform like theirs someday (probably not any time soon) but I don't think I'd do any of what they're doing -- it feels unnecessary, because I feel like I know lots of projects that have actually already done the hard work for this. I probably only think this because I haven't actually done it, but think it's worth sharing the tech anyway. Bear with me as I recently learned that they use nomad[0] and some of these suggestions are kubernetes projects but I'd love to hear why the following technologies were decided against (if they were):
- kata-containers[1] (it does the whole container -> VM flow for you, automatically) with multiple VMM options[2]
- linuxkit[3] (let's say you didn't go with kata-containers, this is another container->VM path)
- firecracker-containerd[4] (very minimal keep-your-container-but-run-it-as-a-VM)
- kubevirt[5] (if you just want to actually run VMs, regardless of how you built them)
- Ceph[6] for storage -- make LVM pools and just give them to Ceph, you'll get blocks, distributed filesystems (CephFS), and object gateways (S3/Swift) out of it (in the k8s space Rook manages this)
As an aside to all this, there's also LXD, which supports running "system" (user namespace isolated) containers, VMs (somewhat recent[7][8]), live migration via criu[9], management/migration of underlying filesystems, runs on LVM or zfs[10], it's basically all-in-one, but does fall behind in terms of ecosystem since everyone else is aboard the "cloud native"/"works-with-kubernetes" train.
I've basically how I plan to run a service like fly.io if I ever did -- so maybe my secret is out, but I sure would like to know just how much of this fly.io got built on (if any of it), and/or what was turned down.
[0]: https://news.ycombinator.com/item?id=26745514
[1]: https://github.com/kata-containers/kata-containers
[2]: https://github.com/kata-containers/kata-containers/blob/2fc7...
[3]: https://github.com/linuxkit/linuxkit
[4]: https://github.com/firecracker-microvm/firecracker-container...
[5]: https://github.com/kubevirt/kubevirt
[7]: https://discuss.linuxcontainers.org/t/running-virtual-machin...
[8]: https://github.com/lxc/lxd/issues/6205
If we can scan container filesystems for dependencies, or choose languages that let us build containers minimally enough that it's only a binary + static libs, we can start approaching systems that have dependency chains almost fully cataloged.
They're already here!
- CoreOS Container Linux (now owned by Redhat)[0]
- RancherOS[1]
- Kubic[2] (more focused on running Kubernetes, but same idea)
There are also tools like Linuxkit[3] which focus on helping you actually build images that run the containers you want and nothing else @ startup, which is pretty cool I think.
[0]: https://coreos.com/os/docs/latest/
[1]: https://rancher.com/rancher-os/
However, I’m not sure what the difference is here from say https://github.com/linuxkit/linuxkit which also has an example for how to use LinuxKit to build Kubernetes environments https://github.com/linuxkit/kubernetes
There was also an issue with nix and docker compatibility that I ran into w/ nix-docker[0], but I don't remember what it was.
Here are some videos I found in my history back when I was exploring Nix (watched in end of 2017/early 2018 I believe):
https://www.youtube.com/watch?v=YbUPdv03ciI
https://www.youtube.com/watch?v=mIxtBVKo7JE
It's entirely possible my anecdata is too old to be useful. Maybe nix is much easier to use (and use correctly) these days. One example of where Nix should have been able to pick up mindshare is with the same people who value linuxkit[1]. Maybe this is is another marketing/branding/hype/money disparity thing but `nix-build '' -A vm` looks like exactly what linuxkit is being lauded/promoted for being a good tool for, and it's been around for so much longer.
Thanks for mentioning OSv[0], I'd never heard about it before.
I'm excited to see what happens with Nabla, I read the accompanying paper and was pretty excited about the possibilities. Nabla's greatest strength is it's ergonomics, IMO, though rumprun comes close (someone took some time to make dockerized containers for it).
[EDIT] - looking at OSv's (primary?) tool capstan[1], I'm not sure it's quite the same, it seems more like linuxkit[2] than rumprun. Maybe my intuition is wrong, but it seems much more like you're bringing along a stripped down OS than you're writing a program meant to interact with a shared unikernel. The distinction is so fuzzy in my head I'm not sure there is one, but it feels different.
[0]: http://osv.io/
You should check linuxkit https://github.com/linuxkit/linuxkit
EDIT: we have an openbsd developer on the team that built LinuxKit and Docker for Mac, so we should be able to help fix any issues...