What resources do you have for hosting this?

I setup a whisper-asr-api backend this week with gobs of CPU and RAM and an RTX 3090. I’d be interested in making the API endpoint available to you and working on the overall architecture to spread the load, improve scalability, etc.

Let me know!

mayeaux

I'm just running this off of a 2x RTX A6000 server on Vast.ai at the moment, about $1.30/h and then using nginx on another server to reverse proxy it to Vast

Open an issue on the Github repo and we can collab for sure!: https://github.com/mayeaux/generate-subtitles/issues

kkielhofner

Cool - will do!

Through a series of events I'm in the beneficial position of my hosting costs (real datacenter, gig port, etc) being zero and the hardware has long since paid for itself. I'm almost just looking for ways to make it more productive at this point.

mayeaux

Hey, I know the feeling, I felt bad when I had my GPU just sitting there and it's just a little Vast server lol. If you want to use your hardware to run this software I'd be more than happy to help get it setup!

kkielhofner

For what's it worth my approach has been running a tweaked whisper-asr-webservice[0] behind traefik behind Cloudflare. Traefik enables end to end SSL (with Cloudlare MITM, I know) and also helps put the brakes on a little so even legitimate traffic that makes it through Cloudflare gets handled optimally and gracefully. I could easily deploy your express + node code instead (and probably will anyway because I just like that approach more than python).

Anyway, I'll be making an issue soon!

[0] - https://github.com/ahmetoner/whisper-asr-webservice