简体繁体 English

如何确定适当的检查间隔？

[英]How do I determine the appropriate check interval?

原文 2010-08-24 17:55:32 3 1 python/ multithreading/ tornado/ gil

I'm just starting to work on a tornado application that is having some CPU issues. 我刚刚开始处理存在一些CPU问题的龙卷风应用程序。 The CPU time will monotonically grow as time goes by, maxing out the CPU at 100%. 随着时间的流逝，CPU时间将单调增加，使CPU达到100％的最大利用率。 The system is currently designed to not block the main thread. 该系统当前被设计为不阻塞主线程。 If it needs to do something that blocks and asynchronous drivers aren't available, it will spawn another thread to do the blocking operation. 如果需要执行某些阻止和异步驱动程序不可用的操作，它将生成另一个线程来执行阻止操作。

Thus we have the main thread being almost totally CPU-bound and a bunch of other threads that are almost totally IO-bound. 因此，我们的主线程几乎完全受CPU约束，而一堆其他线程几乎完全受IO约束。 From what I've read, this seems to be the perfect way to run into problems with the GIL. 根据我的阅读，这似乎是解决GIL问题的完美方法。 Plus, my profiling shows that we're spending a lot of time waiting on signals (which I'm assuming is what __semwait_signal is doing), which is consistent with the effects the GIL would have in my limited understanding. 另外，我的分析显示，我们在信号上花了很多时间（我假设这是__semwait_signal所做的事情），这与GIL在我有限的理解中所产生的影响是一致的。

If I use sys.setcheckinterval to set the check interval to 300, the CPU growth slows down significantly. 如果我使用sys.setcheckinterval将检查间隔设置为300，则CPU的增长会大大减慢。 What I'm trying to determine is whether I should increase the check interval, leave it at 300, or be scared with upping it. 我要确定的是应该增加检查间隔，将其保留为300还是害怕增加检查间隔。 After all, I notice that CPU performance gets better, but I'm a bit concerned that this will negatively impact the system's responsiveness. 毕竟，我注意到CPU性能会提高，但是我有点担心这会对系统的响应能力产生负面影响。

Of course, the correct answer is probably that we need to rethink our architecture to take the GIL into account. 当然，正确的答案可能是我们需要重新考虑我们的体系结构以考虑到GIL。 But that isn't something that can be done immediately. 但这不是立即可以完成的事情。 So how do I determine the appropriate course of action to take in the short-term? 那么，如何确定短期内应采取的适当措施？

1 个解决方案

The first thing I would check for would be to ensure that you're properly exiting threads. 我要检查的第一件事是确保您正确退出线程。 It's very hard to figure out what's going on with just your description to go from, but you use the word "monotonically," which implies that CPU use is tied to time rather than to load. 仅凭您的描述很难弄清楚到底发生了什么，但是您使用了“单调”一词，这意味着CPU的使用与时间相关，而不是与负载相关。

You may very well be running into threading limits of Python, but it should vary up and down with load (number of active threads,) and CPU usage (context switching costs) should reduce as those threads exit. 您可能会遇到Python的线程限制，但是它会随着负载（活动线程数）而上下变化，并且随着这些线程的退出，CPU使用率（上下文切换成本）应降低。 Is there some reason for a thread, once created, to live forever? 创建线程后，存在永久性的某些原因吗？ If that's the case, prioritize that rearchitecture. 如果是这种情况，请优先考虑该架构。 Otherwise, short term would be to figure out why CPU usage is tied to time and not load. 否则，短期将是弄清楚为什么CPU使用率与时间相关联而不是与负载无关。 It implies that each new thread has a permanent, irreversible cost in your system - meaning it never exits. 这意味着每个新线程在您的系统中都有永久的，不可逆的成本-这意味着它永远不会退出。