I've done A LOT of research and experimentation in this space over the course of the last 5-ish years, mainly out of sheer curiosity. This sounds very, very similar to something I prototyped a while back using x264 + Broadway.js (HTML5 video streaming with low latency is a non-starter).

I'm extremely curious, has Paperspace actually developed an entirely new video codec, encoder, and efficient JS decoder for this?

I'm a little confused by the statements so far (and the comments are flowing in, so apologies if this was addressed while I was typing): "we are using a video stream", "using a JS renderer", "building a streaming protocol", "using GPU tech originally developed for video game streaming", "a remote desktop protocol that could stream HD video"

So, is just the streaming protocol you've developed and not the codec? In my expiriments, simply pushing out h264 NAL units over websockets and passing them to the decoder was a pretty solid start. Add a tiny layer of buffering over that and I imagine it'd be fairly stable. Ultimately, I backed away from h264 for licensing and performance issues.

Also, what's the transport into the browser? Websockets? WebRTC data channels? Have you encountered performance issues with Firefox not being able to handle websockets or data channels in web workers? (Which is seemingly coming in FF37.)