What does HackerNews think of firecracker?

Secure and fast microVMs for serverless computing.

Language: Rust

#1 in React
#4 in Rust
#1 in Minecraft
#1 in Serverless
Hey HN,

We've been building cloud for AI agents/copilots/AI apps. One of the first products that we're launching are AI Playgrounds. It's essentially a sandboxed VM (Firecracker [0]) where your agent can freely run code, use filesystem, use headless browser, and more. Every playground has also it's own URL. So for example you can have your agent start a webapp, let the agent work on it inside the playground, and because the playground has URL, the user can see all of the changes.

We've seen that it's especially useful for coding copilots and agents. They can easily run linters, formatters, or debug code and feed all this information back to LLM to make the next iteration better. Our short-term goal is to offer coding agents/copilots all of the tooling that human developers get in their coding editors. And have all of this happen in a secure sandboxed environment that people will eventually be able to self-host it.

You don't need to manage any infrastructure. You just call a single line of code with our SDK and we spin up a cloud playground in a few hundred ms for your AI app.

We have parts of our product open source here https://github.com/e2b-dev and are thinking of actually open-sourcing the whole infrastructure and make it really easy to 1-click deploy for anyone. One of the reasons we haven't done that is how much tooling we had to build on top Firecrackers and how much specialized environment they need. Which makes it a bit of challenge to offer great DX.

Happy to answer more questions!

[0] https://github.com/firecracker-microvm/firecracker

If you're interested in learning more about that, check out our NSDI'20 paper on how we chose this direction (https://www.usenix.org/conference/nsdi20/presentation/agache) and the Firecracker source and docs (https://github.com/firecracker-microvm/firecracker).

Thanks to KVM, and to the minimal hardware support (no PCI, no ACPI, etc), Firecracker's source is rather simple and even relatively readable for non-experts.

systemd-nspawn, LXC, and podman should all be able to do that (though doing recursive containers can be kind of weird). In theory https://github.com/firecracker-microvm/firecracker should as well, it runs very lightweight VMs.
The Firefox repo is 27% C++, 14% C, and 9.5% Rust - https://4e6.github.io/firefox-lang-stats/ AWS wrote Firecracker which powers AWS Lamba - https://github.com/firecracker-microvm/firecracker Cloudflare just replaced nginx with an in-house reverse proxy written in Rust called Pingora (not yet open source) Facebook rewrote Mercurial in Rust to help scale their monorepo (called Mononoke, now part of their EdenSCM project) Dropbox rewrote their sync client in Rust, and they wrote their block storage service (Magic Pocket) in it. A significant portion of Google's new Fuchsia OS is written in Rust, and it is currently powering Nest devices NPM's backend is written in rust curl can now use Rust's hyper library for making http requests
If we were to support more runtimes, I think I'd go for Firecracker (https://github.com/firecracker-microvm/firecracker/) which is what AWS uses for lambdas
I don't write web services, so it's hard for me to answer about axum, but if I were to speculate, I'd point out that PHP, Ruby and Python have good integrations with tools like Apache and nginx, while AOT compiled languages like C++ and Rust do not. There is CGI/FastCGI, but then you're parsing strings in true Unix fashion. When instead you use a web server library written in the language you use, you get to use a richer interface than functions [byte] -> [byte]. Go is nicer here since it has that in the standard library, but the idea is the same. But again: take that with a grain of salt --- I don't write web services.

> I mean i only asked for an example of a small team performing magic thanks to rust so if there's no examples of that, i can be certain there's no examples of a large team performing magic thanks to rust!

Well then, my examples of large teams should be sufficient. :)

Oh, and I've just remembered about this nice project: <https://github.com/firecracker-microvm/firecracker/>

As additional datapoint, afaic(r|s) amazon doesn’t allow any non aws employee maintainers on any repo in a github organization with aws or amazon in the name.

Also freertos was a bit of an acquihire originally of Richard Barry - https://en.wikipedia.org/wiki/FreeRTOS

What I’d like to see more contribution / participation from Amazon around the extant open source community projects they run as saas (postgres, MySQL, Kafka, etc). The numbers here are extremely low.

Also your article seems to have missed firecracker vm, https://github.com/firecracker-microvm/firecracker

Off the top of my head…

Firecracker, the software AWS Lambda runs on, is written in Rust: https://github.com/firecracker-microvm/firecracker

Mozilla wrote Servo, a browser engine, in Rust. Part of it has been integrated into Firefox a while ago:

> Mozilla incorporated the Servo CSS Style engine in release 57 of its Firefox Quantum browser.

https://research.mozilla.org/servo-engines/

It’ll be used in Android: https://security.googleblog.com/2021/04/rust-in-android-plat...

It’ll probably find it’s way into the Linux kernel: https://news.ycombinator.com/item?id=29485465

> I am under the impression that even _safe_ Rust is really hard to learn.

[...]

> The syntax seems overcomplicated, the compiler errors are cryptic, the IDE is not helpful.

Yes, Rust is hard to learn. Rust does _seem_ over-complicated.

However I like to compare Rust to exercise. You need to do a bit of it before you start reaping the benefits of it.

If you suspend your judgement for a bit and try to write some Rust, starting from the very beginning you will find that:

- Rust is actually a small language at its core, unlike the monstrosity that is C++ . You don't really need Advanced Rust to be productive. Use Advanced Rust only when you're... advanced

- Rust actually is very consistent

- The Rust compiler is actually very helpful. It's the least cryptic compiler I've met. But its OK if you feel that now as you're just beginning your journey with Rust

Avoid the temptation to "read" Rust from a book. Try to _do_ Rust. Otherwise it might overwhelm you. Simply keep adding Rust techniques to your arsenal as you mature in your usage of Rust.

Learning Rust changed the way I look at programing. Rust is a beautiful language. As a random example, just look at the the Firecracker VMM written in Rust -- https://github.com/firecracker-microvm/firecracker . It would have been able to very difficult for me to understand the codebase if it were written in C/C++!

Rust is one of those rare languages I've encountered that if the code compiles, there is a high probability it will work. The type system is that good!

TL;DR Persist and you will reap the rewards with Rust.

The site seems light on details. As a security guy, I'm particularly interested in (and a skeptic of) claims of "de-privileging" a Linux kernel. Firecracker[1] micro VMs are very fast, "feel" like containers, and don't depend on modifying the guest OS for security. Plus they're FOSS. This is what I'd measure Kwarantine against; you need to be in the ballpark on all these characteristics and knock it out of the park on manageability to interest me.

[1] https://github.com/firecracker-microvm/firecracker

> Lambda uses containers vs. cloudflare workers use v8 isolates. v8 Isolates are much much faster and more secure for serverless functions.

You're right that v8 Isolates are blazing fast, but Lambda runs functions in a microvm spawn by Firecracker [0], which is likely to be more, not less, secure than Isolates [1].

[0] https://github.com/firecracker-microvm/firecracker/

[1] https://fly.io/blog/sandboxing-and-workload-isolation/

This is awesome. How does this compare with Firecracker - https://github.com/firecracker-microvm/firecracker

Why only TCP is available? Network stack needs to be implemented?

> KVM and Xen have a smaller attack surface and are known to have a better isolation model, but I'm sure there are still plenty of exploits given enough resources.

KVM - the low-level kernel-level hypervisor - has a tiny attack surface and has been audited exhaustively. It's unlikely to have critical bugs in it.

When people talk about "KVM vulnerabilities", they're usually talking about vulnerabilities in QEMU, which implements the actual device emulation. QEMU has all of the attack surface, deals with low-level data shuffling, and is written in C. Even worse, most stock QEMU-KVM deployments simply run qemu as root with no extra sandboxing or MAC like SELinux/sVirt. It's very likely that a bunch of 0days exist for those environments.

This is why many cloud providers use KVM-the-kernel-module, but an in-house replacement for QEMU.

Fortunately, there's a growing ecosystem of QEMU replacements written in Rust:

- https://github.com/cloud-hypervisor/cloud-hypervisor

- https://github.com/firecracker-microvm/firecracker

- https://chromium.googlesource.com/chromiumos/platform/crosvm... (the Chrome OS VM runtime which Firecracker was forked from)

Google's gVisor - the sandbox that App Engine and Cloud Run uses - uses KVM as well: https://gvisor.dev/docs/

With an emulation layer written in a language like Rust, the trust boundary is much better.

As for VirtualBox in particular - that one should not be considered a trust boundary. Nobody is seriously using it in production, and it's regularly featured in CTF competitions as a fun exploitation target.

Firecracker has 13K stars on Github [1]. I'm not in this space. Why does a toy have so much activity (issues, pull requests) and stars?

1 - https://github.com/firecracker-microvm/firecracker

Are you talking about this project? https://github.com/firecracker-microvm/firecracker containerd

I think it's still very much active. Not sure how usable it is if that's what you mean.

I'd hardly call for example firecracker a "pet project by mismanaged megacorp". It's running production workloads - and at AWS scale, I'm fairly certain it's running more work than most of us here ever touched in their career total. https://github.com/firecracker-microvm/firecracker

> And they at least have PEPs.

Rusts language development has been guided by RFCs since like the beginning. There's no comprehensive standard as in "somebody went and wrote down the current state", but it's also not the case that there's no written spec of how things should behave. You could base a new compiler on that and there's actually developments to have different backends to rustc. Cranelift for example https://github.com/bytecodealliance/wasmtime/blob/main/crane...

> "The Rust development team isn't interested in supporting your obscure embedded architecture" or "sorry, this platform was phased out due to technical debt" is what I mean by not serious.

So any compiler team that stops supporting some sort of architecture because it's either rare or no longer worth the effort is not serious in your opinion? Like GCC, that has a decision making matrix for that? https://gcc.gnu.org/backends.html And dropped various architectures along the way?

I'm also biased here, but it seems like Firecracker (https://github.com/firecracker-microvm/firecracker) and rust-vmm (https://github.com/rust-vmm) would be great things to use as examples in a course like this. Not because it has space to explain the whole virtualization stack (it doesn't), but because things like virtualized devices are both fairly simple (at least at the simple end) and really safety critical.
If people are really serious about writing virtualization in Rust, they should just contribute to firecracker: https://github.com/firecracker-microvm/firecracker
I am from AWS. Could we please change the title to say “Bottlerocket from AWS“. Like Firecracker [1] it’s explicitly not AWS branded.

1. https://github.com/firecracker-microvm/firecracker

This project started off as a side-project on secure code execution because I wanted to figure out Repl.it and HackerRank worked. I tried experiments with different technologies like Firecracker [1], AWS Lambda [2] and gVisor [3]. They are all great technologies but I ended up with a solution using a combination of VMs and Docker containers to execute user generated code.

[1] https://github.com/firecracker-microvm/firecracker

[2] https://aws.amazon.com/lambda

[3] https://github.com/google/gvisor

It looks like that's missing data - including some very big ones. https://github.com/firecracker-microvm/firecracker has quite a few stars and came out in 2018 - no mention. All three of them have contributed to the Linux kernel in significant fashions, and it's not listed for any of them, despite the 72k stars.

And are github repos and stars the only way to measure open source contribution? The Xen project only has 162 stars, yet I don't think anyone would say it isn't an important project, and it's one Amazon has contributed to - if you're using a PV guest on Xen that's protected against Meltdown, chances are high you're using the Vixen mitigation that Amazon released.

Could Amazon do more? Probably. I'm not trying to say they're perfect. But this idea that they don't contribute back to open source seems pretty flawed, even by the data in your link, which seems incomplete.

Amazon just open sourced the hypervisor used for Lambda and Fargate ( https://github.com/firecracker-microvm/firecracker )

When the Spectre/Meltdown hit, Amazon also open sourced the mitigation used for the AWS version of Xen ( http://xenbits.xen.org/xsa/xsa254/README.vixen )

If you search the Linux git commits for people with @amazon emails, you'll find contributions, bug fixes, etc.

I don't think it's fair to say that barely anything is contributed back.