networking not an option?
I'm doing that atm, using jack on both boxes, but it requires waay more configuration and fiddling to get working well.
a physical usb connection would provide an intuitive, default-supported way to transfer the data, the audio class supports everything we need for making this work.
I've looked at a couple of chips and ordered two usb -> i2s chips along with a pair of high-speed optoisolators, seems I need a bit of firmware to configure the chips via i2c, but that I can do from an arduino. Just seems such an intuitive thing to have I can't imagine I really have to do it myself.
Take a look at AoIP, AES67 and Dante.
While they're IP and not USB, they are very much designed for extremely low latency for professional applications.