It's not obvious to me from the blogpost where TLS termination happens in this scenario.

I would want it to happen on my local machine, so that (a) Cloudflare can't read my plaintext traffic, and (b) I can manage subdomain certificates more easily via Caddy.

Is that possible with the cheapo free tunnels or does Cloudflare want to handle the domain and TLS certificates, too?

Cloudflare Tunnel doesn't offer an end-to-end encryption option. If this is a must for you, either my own boringproxy or remotemoe[0] both offer this. I'm sure at least a couple others on the list[1] do as well but you'd have to check them individually. If you find any that do please consider opening an issue so I can add that information to the list.

[0]: https://github.com/fasmide/remotemoe

[1]: https://github.com/anderspitman/awesome-tunneling