[英]Where does the limitation of the thread pool come from?
Using the following code, I found that the maximum size of the multiprocessing.pool.ThreadPool
is 11689 (on my machine). 使用以下代码,我发现
multiprocessing.pool.ThreadPool
的最大大小为11689(在我的机器上)。 If I make it bigger, I get 如果我加大它,我得到
RuntimeError: can't start new thread
RuntimeError:无法启动新线程
Can somebody explain where this comes from? 有人可以解释它的来历吗? 11689 seems like a weird system constant... so maybe I ran out of a resource?
11689似乎是一个奇怪的系统常量,所以也许我用光了资源?
Please note: This is not about how to choose the best number of threads for a Thread pool. 请注意:这与如何为线程池选择最佳线程数无关。 This question is about where the 11689 comes from?
这个问题是关于11689的来源?
A brief look at the CPython source code handling threads suggests the interpreter is not delving into details too much when a thread creation fails. 简要了解CPython源代码处理线程,这表明在线程创建失败时,解释器不会过多地研究细节 。 It just raises the RuntimeError you see.
它只会引发您看到的RuntimeError 。
According to pthread_create
manpage, a thread creation can fail with the following error codes and reasons. 根据
pthread_create
联机帮助页,线程创建可能由于以下错误代码和原因而失败。
EAGAIN Insufficient resources to create another thread.
EAGAIN资源不足,无法创建另一个线程。
EAGAIN A system-imposed limit on the number of threads was encountered.
EAGAIN遇到了系统施加的线程数限制。 There are a number of limits that may trigger this error: the RLIMIT_NPROC soft resource limit (set via setrlimit(2)), which limits the number of pro‐ cesses and threads for a real user ID, was reached;
有许多限制可能触发此错误:达到了RLIMIT_NPROC软资源限制(通过setrlimit(2)设置),该限制限制了实际用户ID的过程和线程数。 the kernel's system-wide limit on the number of processes and threads, /proc/sys/kernel/threads-max, was reached (see proc(5));
达到了内核在系统范围内对进程和线程数的限制,即/ proc / sys / kernel / threads-max(请参阅proc(5)); or the maximum number of PIDs, /proc/sys/kernel/pid_max, was reached (see proc(5)).
或已达到PID的最大数量/ proc / sys / kernel / pid_max(请参阅proc(5))。
...
...
My blind guess is that you are incurring in the first problem rather than the second. 我的盲目猜测是,您是在引发第一个问题,而不是第二个问题。 You might be running out of memory to allocate new threads.
您可能会用尽内存来分配新线程。
A deeper explanation on ways a thread creation can fail can be found at this link . 在此链接中可以找到有关线程创建失败的更深入的解释。
This is a blind guess though. 不过,这是一个盲目的猜测。 A more verbose error handling from the interpreter would surely help.
解释器进行的更详细的错误处理肯定会有所帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.