Avoid VirtualBox and becoming trapped in the non-free & personal use only extension by using one of the very decent open source alternatives Boxes[1] or virt-manager[2]
I do not understand, Virtual Box is free software, the versions distributed through most common Linux distributions does not contain any non-free components and the binaries on the site are released under the GPLv2. The extensions you mention are a separate download that you explicitly have to download and it is mentioned around the download that they are under a different license.
How exactly are you trapped by using GPL software?
Without the extension VirtualBox VMs are pretty much shit.
For tools like Vagrant, not at all. They are very useful if you want to attach USB devices directly to the VM, use the built-in webcam and so on, but I've used VirtualBox for years without installing the extension once.
USB 2.0 and 3.0 are not fully supported without the extension pack.
Which again are irrelevant for the common “just for Vagrant” use case.