[英]Creating a warc record with requests.get() response using warcio
I'm using the warcio
library to read and write warc
files. 我正在使用warcio
库读取和写入warc
文件。
When trying to write a record of a response object from requests.get(URL,stream=False)
, warcio
is writing only HTTP headers to the record but not the payload. 当尝试从warcio
requests.get(URL,stream=False)
写入响应对象的记录时, warcio
仅将HTTP标头写入记录中,而不将有效负载写入。 However, when stream mode is enabled it works fine. 但是,启用流模式后,它可以正常工作。
Is there a way store the payload when stream mode is not enabled? 未启用流模式时,是否可以存储有效载荷?
I've found a workaround but not sure if it's the correct way. 我找到了一种解决方法,但不确定这是否正确。 Instead of making request object streamable, I've made the payload streamable BytesIO(response.text.encode())
and this seems to work. 我没有使请求对象成为可流式传输,而是使有效负载可流式BytesIO(response.text.encode())
,这似乎可行。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.