Another similar Pi-based KVM is TinyPilot: https://mtlynch.io/tinypilot/
The author was able to use the MJPEG output of the USB HDMI capture devices to get low latency video.
According to the blog post, TinyPilot also uses µStreamer (from this Pi-KVM project) to achieve its low latency. The initial revision of TInyPilot couldn't get low enough latency using the HDMI over IP extender otherwise.