[英]Keep aiohttp session alive
I'm trying to visit a website every X seconds with parallel and separate sessions, then analyse what is in the response to see if each session should continue or not. 我试图每隔X秒访问一个网站,同时进行并行且独立的会话,然后分析响应中的内容,以查看每个会话是否应该继续。 However, once the code reaches the second loop it fails.
但是,一旦代码到达第二个循环,它就会失败。
import asyncio
from aiohttp import ClientSession
import logging
import time
interval = 30
instances = 2
visit_url = 'http://www.example.org'
tasks = []
logging.basicConfig(
format='%(asctime)s.%(msecs)03d %(message)s', # Log in format time.milliseconds {message}
level=logging.INFO, # Use with logging.info()
datefmt='%H:%M:%S') # Display time as Hours:Minutes:Seconds
class StopException(Exception):
pass
async def quit_app(session, task_, reason):
logging.info("[{}] {}.".format(task_, reason))
session.cookies.clear() # Reset cookies
session.headers.clear() # Reset headers
session.close() # End HTTP connection
raise StopException
async def get_status(response):
if "abow" in response:
return "success"
elif "odoap" or "daoscp" in response:
return "waiting"
elif "nullt" in response:
return "fail"
elif "issue" in response:
return "banned"
elif "pending" in response:
return "pending"
else:
return "pending"
async def initialise(headers):
session = ClientSession()
task_ = len(asyncio.Task.all_tasks()) - instances - 1
passed = False
while passed is False:
async with session as session:
async with session.get(visit_url, headers=headers, allow_redirects=True) as initial:
status = await get_status(await initial.text()) # Check HTML for status
if status == "success":
logging.info("[{}] {}.".format(task_, "Success"))
passed = True
elif status == "pending":
logging.info("[{}] {}.".format(task_, "Pending.."))
await asyncio.sleep(interval)
elif status == "waiting":
logging.info("[{}] {}.".format(task_, "Waiting..."))
await asyncio.sleep(interval)
elif status == "banned":
await quit_app(initial, task_, "Banned")
elif status == "fail":
await quit_app(initial, task_, "Failed")
if __name__ == "__main__":
headers = {
'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
'accept-encoding': 'gzip, deflate, br',
'accept-language': 'en-US,en;q=0.9',
'upgrade-insecure-asks': '1',
'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
} # Add appropriate headers
start = time.clock()
loop = asyncio.get_event_loop()
for i in range(instances):
task = asyncio.ensure_future(initialise(headers))
tasks.append(task)
loop.run_until_complete(asyncio.wait(tasks))
end = time.clock()
print("Process took {0:.4f} seconds.".format(end - start))
This code returns the following error: 此代码返回以下错误:
13:56:58.604 Task exception was never retrieved future: Task finished coro= initialise() done, defined at C:/Users/x/PycharmProjects/tests/src/aiohttp_main.py:49 exception=RuntimeError('Session is closed',) RuntimeError: Session is closed
13:56:58.604以后再也不会检索到任务异常:任务完成coro =初始化()完成,在C:/Users/x/PycharmProjects/tests/src/aiohttp_main.py:49处定义:Exception = RuntimeError('会话已关闭' ,)RuntimeError:会话已关闭
I just can't work out how to make the sessions stay alive until I .close() them... 在我关闭.close()它们之前,我只是无法弄清楚如何使这些会话保持活动状态。
I just can't work out how to make the sessions stay alive until I .close() them...
在我关闭.close()它们之前,我只是无法弄清楚如何使这些会话保持活动状态。
The async with
is a request to close the session. async with
是关闭会话的请求。 Which means that after you write: 这意味着您写完之后:
async with session as session:
...you can no longer use session
once the body of async with
is done executing. ...
async with
主体执行完毕后,您将无法再使用session
。 This is not specific to aiohttp, it's how with
works in Python. 这是不特定于aiohttp,它是如何
with
Python中的作品。 For example, when working with files, with
is a request to close the file: 例如,当处理文件时,
with
是关闭文件的请求:
with open('data.csv') as fileobj:
# ... read stuff from fileobj
# outside the "with" block, fileobj is closed and you
# can no longer read from it
The fix is simple enough, just move the with
outside the while
loop. 修复非常简单,只需在
while
循环外移动with
。 For example: 例如:
async def initialise(headers):
async with ClientSession() as session:
# the rest of the code, including the `while` loop, here
On an unrelated note, you probably want to replace len(asyncio.Task.all_tasks())
with a global counter of your own. 无关紧要的是,您可能要用自己的全局计数器替换
len(asyncio.Task.all_tasks())
。 Using Task.all_tasks()
in this way can start producing incorrect results if you later incorporate other unrelated tasks into the event loop (or a third-party library does that for you). 如果稍后将其他不相关的任务合并到事件循环中(或由第三方库来完成
Task.all_tasks()
, Task.all_tasks()
这种方式使用Task.all_tasks()
可能会开始产生错误的结果。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.