簡體   English   中英

使aiohttp會話保持活動狀態

[英]Keep aiohttp session alive

我試圖每隔X秒訪問一個網站,同時進行並行且獨立的會話,然后分析響應中的內容,以查看每個會話是否應該繼續。 但是,一旦代碼到達第二個循環,它就會失敗。

import asyncio
from aiohttp import ClientSession
import logging
import time

interval = 30
instances = 2
visit_url = 'http://www.example.org'

tasks = []

logging.basicConfig(
    format='%(asctime)s.%(msecs)03d %(message)s',  # Log in format time.milliseconds {message}
    level=logging.INFO,  # Use with logging.info()
    datefmt='%H:%M:%S')  # Display time as Hours:Minutes:Seconds


class StopException(Exception):
    pass


async def quit_app(session, task_, reason):
    logging.info("[{}] {}.".format(task_, reason))
    session.cookies.clear()  # Reset cookies
    session.headers.clear()  # Reset headers
    session.close()  # End HTTP connection
    raise StopException


async def get_status(response):
    if "abow" in response:
        return "success"
    elif "odoap" or "daoscp" in response:
        return "waiting"
    elif "nullt" in response:
        return "fail"
    elif "issue" in response:
        return "banned"
    elif "pending" in response:
        return "pending"
    else:
        return "pending"


async def initialise(headers):
    session = ClientSession()
    task_ = len(asyncio.Task.all_tasks()) - instances - 1
    passed = False
    while passed is False:
        async with session as session:
            async with session.get(visit_url, headers=headers, allow_redirects=True) as initial:
                status = await get_status(await initial.text())  # Check HTML for status
                if status == "success":
                    logging.info("[{}] {}.".format(task_, "Success"))
                    passed = True
                elif status == "pending":
                    logging.info("[{}] {}.".format(task_, "Pending.."))
                    await asyncio.sleep(interval)
                elif status == "waiting":
                    logging.info("[{}] {}.".format(task_, "Waiting..."))
                    await asyncio.sleep(interval)
                elif status == "banned":
                    await quit_app(initial, task_, "Banned")
                elif status == "fail":
                    await quit_app(initial, task_, "Failed")


if __name__ == "__main__":
    headers = {
        'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
        'accept-encoding': 'gzip, deflate, br',
        'accept-language': 'en-US,en;q=0.9',
        'upgrade-insecure-asks': '1',
        'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36'
    }  # Add appropriate headers
    start = time.clock()
    loop = asyncio.get_event_loop()
    for i in range(instances):
        task = asyncio.ensure_future(initialise(headers))
        tasks.append(task)
    loop.run_until_complete(asyncio.wait(tasks))
    end = time.clock()
    print("Process took {0:.4f} seconds.".format(end - start))

此代碼返回以下錯誤:

13:56:58.604以后再也不會檢索到任務異常:任務完成coro =初始化()完成,在C:/Users/x/PycharmProjects/tests/src/aiohttp_main.py:49處定義:Exception = RuntimeError('會話已關閉' ,)RuntimeError:會話已關閉

在我關閉.close()它們之前,我只是無法弄清楚如何使這些會話保持活動狀態。

在我關閉.close()它們之前,我只是無法弄清楚如何使這些會話保持活動狀態。

async with是關閉會話的請求。 這意味着您寫完之后:

async with session as session:

... async with主體執行完畢后,您將無法再使用session 這是不特定於aiohttp,它是如何with Python中的作品。 例如,當處理文件時, with是關閉文件的請求:

with open('data.csv') as fileobj:
    # ... read stuff from fileobj

# outside the "with" block, fileobj is closed and you
# can no longer read from it

修復非常簡單,只需在while循環外移動with 例如:

async def initialise(headers):
    async with ClientSession() as session:
        # the rest of the code, including the `while` loop, here

無關緊要的是,您可能要用自己的全局計數器替換len(asyncio.Task.all_tasks()) 如果稍后將其他不相關的任務合並到事件循環中(或由第三方庫來完成Task.all_tasks()Task.all_tasks()這種方式使用Task.all_tasks()可能會開始產生錯誤的結果。

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM