[英]How can I send an HTTP request from my FastAPI app to another site (API)?
我正在嘗試使用以下代碼片段一次向服務器http://httpbin.org/uuid
發送 100 個請求
from fastapi import FastAPI
from time import sleep
from time import time
import requests
import asyncio
app = FastAPI()
URL= "http://httpbin.org/uuid"
# @app.get("/")
async def main():
r = requests.get(URL)
# print(r.text)
return r.text
async def task():
tasks = [main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main(),main()]
# print(tasks)
# input("stop")
result = await asyncio.gather(*tasks)
print (result)
@app.get('/')
def f():
start = time()
asyncio.run(task())
print("time: ",time()-start)
我正在使用帶有 Asyncio 的 FastAPI 來實現大約 3 秒或更短的最短時間,但使用上述方法,我得到的總時間為 66 秒,超過一分鍾。 我還想保留main
函數以在r.text
上進行其他操作。 我知道要達到如此短的時間,需要並發,但我不確定我在這里犯了什么錯誤。
requests
是一個同步庫。 您需要使用基於asyncio
的庫來異步發出請求。
httpx
通常在 FastAPI 應用程序中用於請求外部服務。 它提供了同步和異步客戶端,可以適當地用於def
和async def
路徑操作。 也推薦用於應用程序的異步測試。 我建議默認使用它。
from fastapi import FastAPI
from time import time
import httpx
import asyncio
app = FastAPI()
URL = "http://httpbin.org/uuid"
async def request(client):
response = await client.get(URL)
return response.text
async def task():
async with httpx.AsyncClient() as client:
tasks = [request(client) for i in range(100)]
result = await asyncio.gather(*tasks)
print(result)
@app.get('/')
async def f():
start = time()
await task()
print("time: ", time() - start)
輸出
['{\n "uuid": "65c454bf-9b12-4ba8-98e1-de636bffeed3"\n}\n', '{\n "uuid": "03a48e56-2a44-48e3-bd43-a0b605bef359"\n}\n',...
time: 0.5911855697631836
如果您願意, aiohttp
也可以在 FastAPI 應用程序中使用。
from fastapi import FastAPI
from time import time
import aiohttp
import asyncio
app = FastAPI()
URL = "http://httpbin.org/uuid"
async def request(session):
async with session.get(URL) as response:
return await response.text()
async def task():
async with aiohttp.ClientSession() as session:
tasks = [request(session) for i in range(100)]
result = await asyncio.gather(*tasks)
print(result)
@app.get('/')
async def f():
start = time()
await task()
print("time: ", time() - start)
如果要限制並行執行的請求數,可以像這樣使用asyncio.semaphore
:
MAX_IN_PARALLEL = 10
limit_sem = asyncio.Semaphore(MAX_IN_PARALLEL)
async def request(client):
async with limit_sem:
response = await client.get(URL)
return response.text
@Alex Noname 很好地使用了異步請求庫。 如果您想讓代碼更快,我建議您使用asyncio.Queue作為替代。 在這個例子中,我創建了 100 個生產者和 100 個消費者。 您可以限制隊列中的最大消息數,例如,生產者等待直到有空間容納新消息
asyncio.Queue(maxsize=100)
我也使用了來自httpx的AsyncClient 。
如果您想了解有關隊列的更多信息,我建議您閱讀這篇文章https://realpython.com/async-io-python/
from time import time
from typing import List
from fastapi import FastAPI
from httpx import AsyncClient
app = FastAPI()
URL = "http://httpbin.org/uuid"
client = AsyncClient()
async def main():
r = await client.get(URL)
return r.text
async def producer(queue: asyncio.Queue):
await queue.put(main)
async def consumer(queue: asyncio.Queue, resp: List):
# await queue.get() == main -> without arguments
resp.append(await (await queue.get())())
async def task():
q = asyncio.Queue(maxsize=100)
response = []
consumers = []
producers = []
[consumers.append(consumer(q, response)) for c in range(100)]
[producers.append(producer(q)) for p in range(100)]
await asyncio.gather(*producers)
await asyncio.gather(*consumers)
print(response)
@app.get('/')
def f():
start = time()
asyncio.run(task())
print("time: ", time() - start)
if __name__ == '__main__':
f()
輸出
['{\n "uuid": "a7713d07-ea5d-40d3-95b4-6673f3c50a8b"\n}\n', '{\n "uuid": "c93f8b89-2c44-40fa-9e5f-736e22ad5f23"\n}\n', '{\n "uuid": "cbb4ad76-7790-45ae-87f1-e425eddc8021"\n}\n', '{\n "uuid": "4c1d81c0-ae7d-401a-99df-e98af3651335"\n}\n', '{\n "uuid": "c5f70738-fbba-4cf9-8fdf-29f8b4eabe63"\n}\n', '{\n "uuid": "d016b852-4312-4502-a336-a6a110237d1d"\n}\n', '{\n "uuid": "22d8b00b-4266-4236-b5a3-ed5d7c5be416"\n}\n', '{\n "uuid": "cd54fdbb-7de9-4df3-90cc-e6b108d5fdf8"\n}\n', '{\n "uuid": "757f0a26-7896-4a04-bea2-60c66a38b05b"\n}\n', '{\n "uuid": "72eb6584-21f4-449b-b6bd-d0f88666126f"\n}\n', '{\n "uuid": "b3deadf5-5b79-491b-829c-0404c306cb68"\n}\n', '{\n "uuid": "789e7422-493d-49d2-9585-e5ca34b7cf36"\n}\n', '{\n "uuid": "48d29a82-ff7c-41f5-8af2-42784326a31f"\n}\n', '{\n "uuid": "84b2d67c-331c-4037-b6e4-c299d93c1899"\n}\n', '{\n "uuid": "386e79f9-073a-4f27-961c-7befcdf95cd4"\n}\n', '{\n "uuid": "8dfdb5e4-dd69-4043-b174-48ec8505f36f"\n}\n', '{\n "uuid": "633e634b-b107-42bb-a7d3-c6bbfff089a0"\n}\n', '{\n "uuid": "962d665f-8663-4be7-a3c6-9426ba500bf4"\n}\n', '{\n "uuid": "320fb858-a751-4c34-9cdb-ddd2f4e28efa"\n}\n', '{\n "uuid": "46a75693-5255-4ac7-8d7a-54910b4d6f68"\n}\n', '{\n "uuid": "5323734b-7ff9-455e-ba5a-66383e6b9a1f"\n}\n', '{\n "uuid": "622a579f-35b6-4e4b-9dba-a8a69c2049c8"\n}\n', '{\n "uuid": "593d5e82-cef3-4be0-99ab-e3034855d7a1"\n}\n', '{\n "uuid": "80f139df-2a27-40c1-8329-e4faa035c45c"\n}\n', '{\n "uuid": "a97e084c-4d30-4c7b-a96e-89ed00dcfe2a"\n}\n', '{\n "uuid": "360d49eb-7222-4064-81c2-6eba2d43a9a5"\n}\n', '{\n "uuid": "a81b6eab-a646-4e58-b986-96a90baa52aa"\n}\n', '{\n "uuid": "0160337e-b400-41d6-ae89-aa46c5131f40"\n}\n', '{\n "uuid": "e600722f-8c15-4959-948b-4c4e5296feb2"\n}\n', '{\n "uuid": "f15403e4-3674-43b2-a0c9-649fd828ba7e"\n}\n', '{\n "uuid": "36bf139c-cc18-45a8-bc55-e7f90ce290b5"\n}\n', '{\n "uuid": "b2368a3c-d86b-4fcd-a0d3-bf7f8f657a83"\n}\n', '{\n "uuid": "d9f16c36-3572-4c70-8a41-3d4e279d76bf"\n}\n', '{\n "uuid": "796087cc-a202-40dd-9921-14802a73323d"\n}\n', '{\n "uuid": "089fa0d7-4c48-4daa-a80d-cb5ebd37dfb7"\n}\n', '{\n "uuid": "e5582bc7-0f8a-4da7-b640-79a0d812154d"\n}\n', '{\n "uuid": "bac0640b-0d0b-4bf2-a3c1-36bdda7cce03"\n}\n', '{\n "uuid": "b4353004-02b2-4846-8692-33dd77ad1d3f"\n}\n', '{\n "uuid": "1b34a744-d0ea-4acf-8bda-33743800d86a"\n}\n', '{\n "uuid": "4d9dd269-6ee2-4356-9bc4-ddf188445320"\n}\n', '{\n "uuid": "a1f380df-0c0d-4aee-bbb7-c3e99fbfe54f"\n}\n', '{\n "uuid": "7cb762eb-1a42-433d-97ea-aa9de4504e35"\n}\n', '{\n "uuid": "981c40e2-64bf-4746-8103-9430bda2a5ca"\n}\n', '{\n "uuid": "22b778eb-82d1-48b9-9874-5ebb80ddb8b1"\n}\n', '{\n "uuid": "e7a9e0e8-7964-400c-aafe-9c36b9b7e1a0"\n}\n', '{\n "uuid": "21a59b91-2732-4bb6-a47e-84008a03c20c"\n}\n', '{\n "uuid": "a78eeb39-5ecb-4509-87c2-b4a2529e3536"\n}\n', '{\n "uuid": "4a332579-ce03-4f69-9db5-78da9196d6b2"\n}\n', '{\n "uuid": "55fbc34f-4eb3-4356-98e3-1df38054a4b2"\n}\n', '{\n "uuid": "257ac454-09c2-4fd4-bdb3-303495360fa2"\n}\n', '{\n "uuid": "7505cc0d-01b3-47f8-91d4-3e54d0f387de"\n}\n', '{\n "uuid": "0fd67af2-622e-4688-b3c8-f64e20f1f3ec"\n}\n', '{\n "uuid": "07653ccf-f408-4807-8ff5-e6098d657451"\n}\n', '{\n "uuid": "b9d0ff18-fd67-4afa-adbe-ebcb53380804"\n}\n', '{\n "uuid": "70d4d53b-2f06-41be-bb38-47f010cfa40f"\n}\n', '{\n "uuid": "a6d49873-e749-4578-ae9c-e6c6f473535d"\n}\n', '{\n "uuid": "e67efee5-76ad-4812-bb97-016ef9ff87e8"\n}\n', '{\n "uuid": "67886926-b2d9-44fb-b836-26b81c53e5fb"\n}\n', '{\n "uuid": "dcbd4ff8-e3cd-4e03-b12d-5fb3834b0e00"\n}\n', '{\n "uuid": "65c2eaee-5fa2-4b58-a1c3-adeb04d92c71"\n}\n', '{\n "uuid": "2cee4ec9-952e-45c5-91b7-f4f5848c3455"\n}\n', '{\n "uuid": "8e94bf1c-ee5a-483a-a962-d0b9aea48c95"\n}\n', '{\n "uuid": "c1fe17bc-bedf-4c4c-952d-a5921f693d9f"\n}\n', '{\n "uuid": "221456fd-48ca-4826-a8b5-5fa0b23db6e4"\n}\n', '{\n "uuid": "62fda759-b382-44e4-ad7d-d19a952fc1c7"\n}\n', '{\n "uuid": "73faeb91-215e-4e49-8f11-11b98e499cc7"\n}\n', '{\n "uuid": "f3279c45-ebcc-4079-b823-3efe825c7cf8"\n}\n', '{\n "uuid": "b892672b-4510-44f4-b61e-9cccaa52421e"\n}\n', '{\n "uuid": "8926979d-71a7-4171-9389-ddafff89e229"\n}\n', '{\n "uuid": "d97cef59-4862-42ca-b0f2-261f98fd4b6f"\n}\n', '{\n "uuid": "3362ff93-89e4-4889-a2f2-2e03771e86ce"\n}\n', '{\n "uuid": "9f525251-4fe4-4a9c-97b5-2f01d2b37aaf"\n}\n', '{\n "uuid": "036959d4-3179-40f9-bbf3-32274f2cede2"\n}\n', '{\n "uuid": "157f8c22-6214-4e27-ab5d-08d39f96d1d3"\n}\n', '{\n "uuid": "e4bfbf62-7c33-4fd7-a231-47f5ce398041"\n}\n', '{\n "uuid": "a41512c1-3346-4457-a379-64d690ffc2ea"\n}\n', '{\n "uuid": "7bb07cfb-294b-44fa-a8dc-6d283c54409f"\n}\n', '{\n "uuid": "f2297d22-a2d0-47ff-8d65-24c6fe7877a7"\n}\n', '{\n "uuid": "645e255b-4c93-4c8f-9ff2-43da293db660"\n}\n', '{\n "uuid": "9190e370-dfa9-47a6-8cef-8df7ab762433"\n}\n', '{\n "uuid": "83216551-9f1b-48b2-8cd6-fd125a7ce965"\n}\n', '{\n "uuid": "aaddb98c-879b-472d-aa39-1a684ef7d179"\n}\n', '{\n "uuid": "4bd7e2fd-1453-4433-aa9f-bc29d82f5b9d"\n}\n', '{\n "uuid": "b02d65e8-2063-4060-96af-088ec497fc10"\n}\n', '{\n "uuid": "e10e3dd2-83c5-4595-afe4-4145bce79193"\n}\n', '{\n "uuid": "8cb62784-1b5d-4dcc-8342-02ad7d417ca9"\n}\n', '{\n "uuid": "13ef1509-4f69-4426-ac42-cb29a2d0f094"\n}\n', '{\n "uuid": "4d4571d5-69bb-4625-b246-b5eef50aa10d"\n}\n', '{\n "uuid": "75e7a2ca-bfa8-43b9-b33a-f3f927453579"\n}\n', '{\n "uuid": "0a8cc8ff-2039-4873-9e38-afad3e10d726"\n}\n', '{\n "uuid": "189ae75b-4879-4897-9725-f9be17e49844"\n}\n', '{\n "uuid": "ba482468-f45f-4060-a0c1-3ef31bb283c8"\n}\n', '{\n "uuid": "3809f1c7-2f11-487d-bf96-8abf64e08298"\n}\n', '{\n "uuid": "da5ea88b-974d-4238-9654-ac56c657c8b4"\n}\n', '{\n "uuid": "edc3de79-7cf4-42a3-a5f4-b754136a6fd3"\n}\n', '{\n "uuid": "6f5ecd91-537c-4009-8435-6c31ce035d36"\n}\n', '{\n "uuid": "4a33b29d-78ba-468f-8f30-a01b3d9e2a87"\n}\n', '{\n "uuid": "a5a2ef2d-d4a2-48e1-8335-f8c1309328c4"\n}\n', '{\n "uuid": "3d1679da-afdd-4f04-9c16-0aaea4c53d0c"\n}\n', '{\n "uuid": "c4025845-0d4c-4549-acb8-1a249b33e644"\n}\n']
time: 1.0535461902618408
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.