I'm trying to get Nix to compile software against host-system provided cuda libraries (and the libc those libraries are dependent on), for a Jetson ARM-based system. So I'm trying to create overrides for these basic libraries based on tar'ed system libraries. Has anyone tried to use Nix this way?

Yes! Although not for CUDA. I used opencl. nixgl helped here: https://github.com/guibou/nixGL. CUDA should be even simpler since you just add the CUDA .so paths to the RPATH. Allthough I might be missing something. Custom libc might be a whole can of worms though if it happens to be incompatible.