Personally, my problem with trying to use Vim keybindings in VS Code is not about the motions of moving around/editing the code itself, but everything else. You end up in plenty of situations where you have to resolve using the mouse because you ended up on some sub-page or whatever, where you either need to resolve using the mouse or keep pressing Tab until you get to where you want to go.
Installing plugins is one good example of this, where even if you use a Vim plugin (or this, it seems), you still need to resolve to using the mouse/Tabbing to actually install the plugin itself.
While with Neo/Vim itself, you never need to reach for the mouse, because the entire thing, not just the editing/writing, is meant to facilitate keyboard-only usage.
If only there was some sane oob setup for neovim with decent file exploring, and debugging.