Why, why do they make it header only? Is it so difficult to integrate a couple source file along with the existing headers?

We should not forget compilation times. A project I use depends on spdlog, a header-only C++ logging library. The thing adds almost two seconds per compilation unit to single threaded build times. And since logging is kinda used everywhere, the whole project takes forever to build (trice the build time it would have had without spdlog, I've measured).

What benefit is so great that it is worth killing compilation times?

The same idea can also be used for C++ headers (unless it's all template interfaces).

With such stb-style headers, you can even get better compile times, because you can include all header-libs into a single implementation source file, giving the same advantages as unity-builds (merging all sources into one file).

PS: origin of the name "stb-style": https://github.com/nothings/stb (although I guess the general idea existed before)