[英]Python: How can i send multiple HTTP requests and receive the response?
How can I send like 1000 requests the fastest way? 如何以最快的方式发送1000个请求? I know that you can send multiple request with
grequests
: 我知道您可以使用
grequests
发送多个请求:
urls = [
'sample.url/1',
'sample.url/2',
...
]
request = (grequests.get(u) for u in urls)
print grequests.map(request)
But the return is not the content. 但回报不是内容。 What I need is to get the json data, so for example something like this:
我需要的是获取json数据,例如:
request = (grequests.get(u) for u in urls)
content = grequests.json(request)
The items returned are not the content, but they do include the content. 返回的项目不是内容,但它们确实包含内容。 You can fetch all of the content like so:
您可以像这样获取所有内容:
result = grequests.map(request)
content = '\n'.join(r.content for r in result) # raw content
text = '\n'.join(r.text for r in result) # decoded content
You can parse the json like this: 您可以像这样解析json:
result = grequests.map(request)
json = [r.json() for r in result]
Sample program: 示例程序:
import grequests
import pprint
urls = [
'http://httpbin.org/user-agent',
'http://httpbin.org/headers',
'http://httpbin.org/ip',
]
requests = (grequests.get(u) for u in urls)
responses = grequests.map(requests)
json = [response.json() for response in responses]
pprint.pprint(json)
text = '\n'.join(response.text for response in responses)
print(text)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.