By the sake of god, please try to add also a more friendly build system too. Bazel is a disaster that would almost make maven looks lightweight, npm reliable and autotools userfriendly.
What do you dislike about bazel? Not championing it, just curious.
The list would be too long but out of order :
- Using a JVM to compile mainly C++ and python, implying to deploy a JDK/JRE just for that.
- Awful RAM consumption, compiling tensorflow takes 16GB of RAM to create a python wheel
- Try to download the world and compile everything without allowing to specify external dependencies.
- Compile time that are crazy long due to previous reason.
- Invasive. Make very hard to integrate with anything else that does not build with Bazel
- Make very hard, or sometimes, impossible to tune compiler flags
- Just not reproducible.
- Unstable, try to build a recent Google package with a 6 month old Bazel and good luck.
I strongly advise you to watch this video of the last FOSDEM : https://archive.fosdem.org/2018/schedule/event/how_to_make_p...
I strongly agree with you on the first one.
Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times.
I do see Bazel seems to support depending on a prebuilt .so, though I have not tried this: https://docs.bazel.build/versions/master/cpp-use-cases.html#...
> Invasive. Make very hard to integrate with anything else that does not build with Bazel
I think your main options here are:
1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#...)
2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/mast...
> Make very hard, or sometimes, impossible to tune compiler flags
"bazel --copt= :target"?
> Just not reproducible.
What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there.
Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need.
> Many of your other ones are byproducts of the fact that Bazel is primarily a build-from-source system. This has some benefits, particularly in a C++ ecosystem where binary compatibility across versions basically doesn't exist. But it also has some big drawbacks when it comes to compile times.
Nix, Guix and Spack packager managers solved the C++ ABI issue a long time ago already without the crazy needs of Bazel in term of resource consumption, integration and compile time. They even supports binary distributions for some of them.
> I think your main options here are: > 1. prebuild the other projects, then depend on the .so from Bazel (https://docs.bazel.build/versions/master/cpp-use-cases.html#...) > 2. write a BUILD file for the external project. Here is an example of a project that builds a bunch of non-Bazel deps by writing Bazel BUILD files for each of them: https://github.com/googlecartographer/cartographer/tree/mast....
I know that. But all of them are terrible options. I do not want to depend of SQLite, OpenSSL, libxml or whatever other system library compiled by Bazel, nor I want Bazel to takes 45 minutes to recompile them. Additionally, this will cause diamond dependency problem with other softwares that use Bazel artefact without compiling with Bazel.
> "bazel --copt= :target"?
Can I use that to specify a flag to some target and not some other ? Without having to build sequentially each of them ?
Concrete example of the Maddness: SQLITE will not compile if you enable some options that would make tensorflow faster.... Bazel recursively compile both.
> What isn't reproducible? I tend to think of reproducibility as a strength of Bazel. Because all of your dependencies are explicit and Bazel fetches a known version, the build is less dependent on your system environment and whatever you happen to have installed there.
Bazel try to build in isolated environment but do half of the job. It still depends on system compiler, and do not chroot nor "compiler-wrap" ( c.f Spack ) making the build still very vulnerable of system side effect and update.
> Disclosure: I am a Googler. I have some gripes with Bazel, but overall I think it gets some important ideas right. You have a BUILD file that is declarative, then any imperative code you need goes into separate .bzl files to define the rules you need.
I can understand that Bazel is very convenient in Google environment with Google resources. But it's a nightmare for everyone I talked to outside of Google.
> Nix, Guix and Spack packager managers solved the C++ ABI issue a long time ago
Yes this is certainly something that is solvable at the package manager level also. And this approach will certainly have shorter compile times.
I agree it would be nice if Bazel integrated with package managers like this more easily. I hope Bazel adds support for this. There is a trade-off though: with less control over the specific version of your dependencies, there is a greater risk of build failure or bugs arising from an untested configuration. Basically this approach outsources some of the testing and bugfixing from the authors to the packagers. But it's a trade-off I know many people are willing to make.
> Can I use that to specify a flag to some target and not some other ? Without having to build sequentially each of them ?
You can put copts=[""] in the cc_library() rules in the BUILD file. This will give per-target granularity. You can add a select() based on compilation_mode if you need to define opt-only flags: https://docs.bazel.build/versions/master/configurable-attrib...
> Bazel try to build in isolated environment but do half of the job. It still depends on system compiler, and do not chroot nor "compiler-wrap" ( c.f Spack ) making the build still very vulnerable of system side effect and update.
Bazel allows you to define your own toolchain. This can support cross-compiling and isolating the toolchain I believe, though I don't have any direct experience with this: https://docs.bazel.build/versions/master/toolchains.html
> I can understand that Bazel is very convenient in Google environment with Google resources. But it's a nightmare for everyone I talked to outside of Google.
I hear you and I hope that we see some improvements to integrate better with package managers.
FWIW, I have been experimenting with auto-generating CMake from my Bazel BUILD file for my project https://github.com/google/upb. My plan is to use Bazel for development but have the CMake build be a fully supported option also for users who don't want to touch Bazel.
Tangentially related to package manager integration: Bazel can now build with CMake in-tree [2].
[0] https://github.com/bazelbuild/rules_nixpkgs/