Wow, so maybe instead of a node_modules folder, npm should use a node_modules.db?

This sounds like a fantastic idea. You would go from 4000 directories and 50000 files to a single file with b-tree indices, hashes, etc. You could probably get a 100x speedup over the current approach.