My understanding is that Meta/Google had to rewrite tons of stuff (distributed file systems, CVS, search tools, build tools...) and need many teams to maintain all these systems, and sometime their developer experience is inferior to what you get with off-the-shelf OSS. I'm not super familiar with this topic as I don't work with systems of that scale, but I'm wondering was it worth it or necessary? what would be the alternative?

If your codebase is "normal-sized," you don't need nearly that amount of infrastructure. There is probably some growing pain when transitioning from normal-sized to "huge," but that's part of the growing pain for any startup. You're going to have to hire people to work on internal tooling anyway; setting up a distributed build and testing service (especially now there are so many open-source and hosted implementations) is worth the effort once you're starting to scale. You're going to have to set that up regardless of a mono-repo or many separate repos.

It's probably only worth hiring serious, dedicated teams that work on building like Google once your CI costs are a significant portion of operation. That probably won't happen for a while for most startups.

https://github.com/bazelbuild/bazel-buildfarm

https://cloud.google.com/build

https://aws.amazon.com/codebuild/

https://azure.microsoft.com/en-us/products/devops/pipelines/