A lot of the speed games get is by having bad load times when starting up.

Keeping startup really fast and all interactions really fast is more challenging than it seems.

This is not the whole story, but one example of the differences in perception.

But games are loading enormous textures, meshes, maps, etc. UIs don’t need any of that.

A mesh may have a lot of triangles, but it's still a single object you can memory map. UIs are typically a bunch of heap allocations and pointer jumping all created at runtime. That can be surprisingly inefficient.

then make it efficient. overallocate a block of memory and use it for all your tiny ui elements. its exactly what games do.

There are game-like UI libraries (https://github.com/ocornut/imgui) but to use them you have to write your program like you're a gamedev and Conway's Law usually fights that.