Considering how ubiquitous it is, it's a mystery how and why there's so little documentation about how to use the library versions libavcodec and libavformat.

I think you're supposed to read the header files? I have no idea how people write ffmpeg stuff. The only good tutorial I've seen is: https://github.com/leandromoreira/ffmpeg-libav-tutorial