So if I understand correctly, I could declaratively specify my workstation, say I want vim, podman, codium, kubectl, curl, tmux, firefox, zettlr. I could put this in a configuration and have my workstation anywhere I can grab that from?

Is there some way I can boot-to-git-configuration? So I could do this in RAM on someone else's hardware, like a network boot?

Alternative to what others have said, you can use your config to build a custom bootable ISO and use it to setup your new system. Comes very handy if you can't guarantee nice network for setup.

This sounds like a great idea. Do you happen to know of any documentation to assist with this? I'd love to create a custom, bootable USB which I can upgrade periodically, keeping my SSH keys on a Yubikey.

It's just a small wrapper but it shows you most of the targets available.