What does HackerNews think of pathos?
parallel graph management and execution in heterogeneous computing
Language:
Python
The models for curve fitting in my lab were using lambdas, which cannot be serialized by multiprocessing/pickle (at least that was the state about 5 years ago). The pathos.multiprocessing module [1] served as a drop-in replacement, which was able to handle lambda with its own dill serializer. Saved me quite a bit of refactoring.
has anyone used ANY object serialization library (other than pickle) with multiprocessing ?
I just cant get anything else to work. We have a sophisticated desktop pyqt application that could really do with a better serialization with multiprocessing.
I have heard of Pathos[1] to replace multiprocessing - but never gave it a try.
A solution to the irritating pickling behaviour of multiprocessing.Pool is to use Pathos.multiprocessing https://github.com/uqfoundation/pathos