My desktop at work has 16G of RAM. I didn't bother setting up swap, and I find the old guidance (2x RAM) pretty absurd at this point. I've had the OOM-killer render the system unresponsive a couple of times, but only because I'd written a program that was leaking memory and I was pushing it to misbehave. If you really want virtual memory on purpose, you can still set up a memory-mapped file for your big data structure.

I don't have swap either. On 8GB it is pretty annoying, because a program I often use frequently overcommits and the system hangs.

Is there any way to tell the OOM killer which program to kill first?

By default, it'll start killing processes when free memory drops below 10%, though you can configure the threshold. I had the same problem for years, and then I started using earlyoom and I don't have to deal with it anymore.