Why isn't the how-docs tested? I don't work in ops or anything like that, but it seems to me that it'd be the best way to ensure everything stays up to date.

In Rust (and many other languages), when you document a function, you usually add a code snippet that explains how to use this function. That code snippet will be treated as a unit test and ran when testing the functionality. In this way, documentation is always up to date (assuming the tests are ran).

Why can't tool documentation be the same? The tool documentation usually contains a sample tool invocation. All we need to do is run the tool in a controlled environment, and ensure that the invocation runs successfully. When a tool gets updated, run the "documentation tests" to ensure all the commands in the docs still run successfully. If they don't, then update the docs. Surely, there are tools out there that already do this? Does anyone know?

It's designed so that you can write tests that are the basis of the "how to" documentation - with a small script and some jinja2 you can generate readable documentation in the form of markdown from the test:

Original test (behavioral example) : https://github.com/crdoconnor/strictyaml/blob/master/hitch/s...

Example docs generated from that test : https://hitchdev.com/strictyaml/using/alpha/scalar/email-and...

It's also designed so that the testing framework can be programmed to rewrite the test automatically for simple code changes (e.g. changing the wording of that exception).

This makes building tests much quicker and much, much less tedious and perfectly in sync docs almost becomes a side effect.

Essentially "how to" documentation and example driven behavioral tests are the same thing for all projects. It's just that most people don't realize and don't treat them as such.