He mentioned partitioning bare-metal into VMs (which is a good idea, containers aren't a great security boundary) but it doesn't look like MRSK does that (yet?).

I can see using this to deploy on DO or Hetzner to save a few bucks vs Render or Amplify- but on bare metal you're giving up a pretty amazing ecosystem these days. Ie. if you want to run stateful things like databases, logging/monitoring, NFS, object storage, persistent disks, complex networking, etc it's going to be miles easier to manage that w/ k8s. Even for that bare-metal/vm-partitioning goal above, there is a k8s runtime (kata-containers) which will automatically run containers on lightweight vms and handle all the plumbing to and from K8s. Plus if your mid-size SaaS wants to sell to the whales you're going to need compliance and security best practices that are much easier to implement and enforce w/ k8s. Finally, there are also lighter weight distributions these days to make it less painful to run yourself.

That said, I do appreciate the 37signals philosophy of having such strong opinions they craft their own tools and I also applaud leaving the cloud for bare metal. Even if you need the burst capacity you can provision your base-load on bare-metal and burst to the cloud when needed.

I had the impression k8s is still not recommended to run a DB or any kind of persistence. Are any large companies doing that in production?

Regarding logging, object storage, how does kubernetes help? You'll have to implement these regardless of the underlying container infra.