python gevent.pool running on 1 thread

Question

my code run a pool of tasks and when I monitor the activity one of the cpu(more like one of the threads) reach 90%, I wish to work more "multi thread" and spread the task between the threads than work on 1 thread.

gevent lib work on a core level and not threads? any other way to create pool and spread it over threads?

group = gevent.pool.Pool(size=18)
for url in urls:
    group.spawn(scrap_content, url)
group.join()

Answer 1

Python is fundamentally single threaded. What gevent does is to give you a better interface to work with async io.

If you need to spread computational intensive work to more than one cpu, you should check the multiprocessing module

Answer 2

Gevent uses greenlets, which are coroutines, not threads. It is absolutely normal to see everything run on only one CPU.

Note that even if you were using threads in Python, you would still see it run on one CPU only because CPython (the standard implementation of Python) does not support multithreading on different CPUs.

The reason for that is the infamous GIL (global interpreter lock). Python's core is not thread safe because of the way it does garbage collection, so it uses a lock, which means threads accessing python objects run one after the other.

If you need to do real multithreading on several CPUs, look at Cython (not to be confused with CPython) and "no_gil", or c-extensions, or the multiprocessing module.