It looks like the author is most bothered by compile times of dependencies.
Cargo needs to do better with shared caches (so you compile each dep at most once per machine) or ability to get precompiled crates (so you don't even compile it).
Incremental improvements of compiler speed or trimming of individual dependencies won't bring the 10x improvement it needs.
Have you tried sccache [0]? It doesn’t always choose to cache a dependency, but it helps about 70% of the time. Anecdotally, it hastened a release build of a pretty standard CLI tool (with incremental compilation) by almost 4x.
In the context of resource-constrained machines, one can always host it remotely on S3. (or mount an NFS share as the CARGO_TARGET_DIR, if you’re feeling adventurous or want fast CI)