The trouble with Pool.map is that the function you pass in has to be picklable, i.e. defined at the top level in the module you are currently in.

So you can't use it in interactive sessions:

  In [1]: import multiprocessing\n  In [2]: pool = multiprocessing.Pool(8)\n  In [3]: def hello(x):\n  ....:     return x+1\n  ....:\n  In [5]: stuff = range(10)\n  In [4]: pool.map(hello, stuff)\n  # cryptic errors\n

\nYou also can't use it with lambdas or functions defined within functions; suppose I put this in test.py:

  import multiprocessing\n  def run_stuff():\n      pool = multiprocessing.Pool(8)\n        def hello(x):\n          return x+1\n      stuff = range(10)\n      return pool.map(hello, stuff)\n\n  print(run_stuff())\n

\n... I get the same errors.

This is sad because pool.map seems to be faster than anything I have written myself which might replace it without the above limitations. Unless someone out there has done better than me? :)

A solution to the irritating pickling behaviour of multiprocessing.Pool is to use Pathos.multiprocessing https://github.com/uqfoundation/pathos