I want something that I can self host. I am perfectly OK with a single language and a few mistakes here and there.

Does such a thing exist? I would gladly donate to a kickstarter project for this before trying to build one myself.

You (essentially) need GPU but here you go:

For your requirements the medium.en model (max) should be satisfactory.

https://github.com/ggerganov/whisper.cpp makes it relatively feasible to run on CPU.