[英]Maximum number of connections per host with twisted.web.client.Agent
I have the following code which creates an HTTPConnectionPool using TwistedMatrix Python framework, and an Agent for HTTP requests: 我有以下代码,该代码使用TwistedMatrix Python框架创建HTTPConnectionPool,以及用于HTTP请求的代理:
self.pool = HTTPConnectionPool(reactor, persistent=True)
self.pool.retryAutomatically = False
self.pool.maxPersistentPerHost = 1
self.agent = Agent(reactor, pool=self.pool)
then I create requests to connect to a local server: 然后我创建请求以连接到本地服务器:
d = self.agent.request(
"GET",
url,
Headers({"Host": ["localhost:8333"]}),
None)
The problem is: the local server sometimes behaves incorrectly when multiple simultaneous requests are made, so I would like to limit the number of simultaneous requests to 1. 问题是:发出多个同时请求时,本地服务器有时会出现不正确的行为,因此我想将同时请求的数量限制为1。
The additional requests should be queued until the pending request completes. 应将其他请求排队,直到待处理的请求完成。
I've tried with self.pool.maxPersistentPerHost = 1
but it doesn't work. 我已经尝试过使用
self.pool.maxPersistentPerHost = 1
但是它不起作用。
Does twisted.web.client.Agent with HTTPConnectionPool support limiting the maximum number of connections per host, or do I have to implement a request FIFO queue myself? 带有HTTPConnectionPool的twisted.web.client.Agent是否支持限制每个主机的最大连接数,还是我必须自己实现请求FIFO队列?
The reason setting maxPersistentPerHost
to 1
didn't help is that maxPersistentPerHost
is for controlling the maximum number of persistent connections to cache per host. 将
maxPersistentPerHost
设置为1
的原因没有帮助,因为maxPersistentPerHost
用于控制每个主机缓存的最大持久连接数。 It does not prevent additional connections from being opened in order to service new requests, it will only cause them to be closed immediately after a response is received, if the maximum number of cached connections has already been reached. 它不会阻止打开其他连接来服务新请求,只会导致已接收到最大数量的高速缓存的连接后,在收到响应后立即将其关闭。
You can enforce serialization in a number of ways. 您可以通过多种方式强制执行序列化。 One way to have a "FIFO queue" is with
twisted.internet.defer.DeferredLock
. 具有“ FIFO队列”的一种方法是使用
twisted.internet.defer.DeferredLock
。 Use it together with Agent
like this: 与
Agent
一起使用,如下所示:
lock = DeferredLock()
d1 = lock.run(agent.request, url, ...)
d2 = lock.run(agent.request, url, ...)
The second request will not run until after the first as completed. 第二个请求要等到第一个请求完成后才能运行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.