this one helps. CPython will not execute bytecode on more than one core at once. Multi-threading cpu-bound code is pointless. The Global Interpreter Lock (GIL) is there to protect all of the reference counts in the process, so only one thread can use Python objects at a time. You are seeing worse performance because you still only have one thread at a time working, but now you are also changing thread contexts.
GDB hangs when multi-threaded python extension is used to debug multi threaded program
may help you . It turns out this is because one of the things I'm passing inside parser_variables is a class (a reader from a third-party module). If I remove the class, the above works fine. For whatever reason, pickle doesn't seem to be able to handle this particular object.
In Python, do I need to protect data transfer between multi-threaded processes?
wish of those help First of al, the threading module uses threads, not different processes! The crucial difference between threads and processes is that the former share an address space (memory), while the latter don't.
Python reading data from CSV in memory and writing it to cassandra in multi threaded