I keep a small series of patches for various software on my system, to fix, or at least relocate, config files to XDG locations[1]. I do this for applications I care about keeping their state. For other applications I either set the environment variables or remove the dirs on login[2]. Of course, the complexity of the project and my familiarity with the language also affects whether I bother with it. Not gonna be able to change ~/.cargo or ~/.dotnet soon (unless someone has patches for those?)
It's buried in my gentoo config, but I wonder if there'd be a good way to make this useful on other distros.
[1]: https://github.com/search?q=repo%3Amid-kid%2Fgentoo-config%2...
[2]: https://github.com/mid-kid/config/blob/master/shell/.zprofil...
Regarding ~/.cargo (and other tools), I've had some success with following suggestions from https://github.com/b3nj5m1n/xdg-ninja