The rust compiler does something similar by testing on all (ish) of crates.io, the public package repository.
yup, they reference it as an inspiration: https://github.com/rust-lang/crater
it's probably impossible to automate an entire ecosystem, and there is value to enabling a tighter integration within a project ecosystem (a subset of the language ecosystem).