Python Multiprocessing vs Eventlet

Question

Based on my understanding, threads cannot be executed in parallel(executed based on availability and random) and thats the reason Eventlet are being used.

If Eventlets are more for parallelism why can't we just use multiprocessing module of Python.

I thought of executing multi process modules and use the join method() to check if all the process are complete.

Can someone explain if my understanding is correct?

Answer 1

Based on my understanding, threads cannot be executed in parallel (executed based on availability and random)

Correct

and thats the reason Eventlet are being used.

Not so correct. The Eventlet library is used to simplify non-blocking IO programming. It does not actually add parallelism. Thread execution is still limited to one thread at a time due to the GIL. But it is used because it greatly simplifies the process of launching, scheduling, and managing IO-bound threads, particularly ones that do not need to interact with each other.

If Eventlets are more for parallelism

As I just mentioned, this is not what they exist for.

why can't we just use multiprocessing module of Python. I thought of executing multi process modules and use the join method() to check if all the process are complete.

You certainly can! And you will get actual parallel execution with this approach. But you may not get the same speedup. The multiprocessing library is better suited for CPU-bound parallel tasks, because those are the ones that need more frequent access to the interpreter. You may actually see an increase in execution time when using multiprocessing with IO-bound tasks because of the overhead of multiple process execution and management.

---

As is the case with most optimization and execution time questions, trying both and profiling is the surefire way to guarantee you're using the "best" option for your application. Though you may find that if you write the code to utilize Eventlets first, then try to modify it to use regular threads or multiprocessing, you'll have to write more boilerplate code just to manage the threads or processes, and the value of Eventlets should become more obvious.