buffalobuffalo

I've never been able to find a good use case for pypy. Why would you ever want to run CPU bound tasks in python? I'm sure there are arguments about the huge ecosystem, not having to rewrite code in another language, etc. But most of the widely used stuff in python already has underlying compiled code for the heavy lifting. Also, given the large amount of parallelism in modern hardware architecture, and python's lackluster concurrency support, I just don't see a reason for using it.

daveFNbuck
I used Luigi [1] to automate data processing at a previous job. It's a simple job queue with a UI. You request jobs from it, and then run them for minutes or hours, so it shouldn't normally be a bottleneck and it makes sense to use a language that's quick and easy to write.

It's written in Python and works fine to process thousands of jobs per day. Once you start having tens of thousands of jobs in the queue, it gets slow enough that it can back things up. This compounds the problem, eventually resulting in the whole thing crashing.

By switching the interpreter to PyPy, I was able to keep the data pipeline running at that scale without having to rewrite anything.

[1] https://github.com/spotify/luigi