[英]How to get cookie from scrapy response and set the cookie to the next request?
I have disabled the Default Scrapy cookie option, so that i have to set it manually.我已经禁用了 Default Scrapy cookie 选项,所以我必须手动设置它。
COOKIES_ENABLED = False
COOKIES_DEBUG = True
Now, i need to set cookie with the value which is received as the response of the same site.现在,我需要使用作为同一站点的响应接收到的值来设置 cookie。 I can able to get the cookie as below,
我可以得到如下的cookie,
cookie = response.headers.getlist('Set-Cookie')[0].split(";")[0].split("=")
now i am trying to set it to the form request by现在我正在尝试将其设置为表单请求
FormRequest.from_response(response,
formdata={"username": "asldkfs", "pass": "slskd"},
cookies={cookie[0]:cookie[1]},
meta = {'dont_redirect': True,'handle_httpstatus_list': [302]},
callback=self.redirection)
def redirection(self,response):
self.log("redirection")
self.log(response.headers)
self.log("Cookie2")
cook1 = response.headers.getlist('Set-Cookie')[0].split(";")[0].split("=")
self.log(cook1)
self.log("end cookie2")
return Request("http://something.net/some/sa/"+response.headers.getlist('Location')[0],cookies={cook1[0]:cook1[1]},
callback=self.check_login_response)
.
.
.
So I could not set the cookie.Do i need to set any other value also or what could be the problem?所以我无法设置cookie。我还需要设置任何其他值还是可能是什么问题?
The cookies argument only works if you have COOKIES_ENABLED set to True, since CookiesMiddleware handles it. cookies 参数仅在您将 COOKIES_ENABLED 设置为 True 时才有效,因为 CookiesMiddleware 会处理它。
Hence you have to set it manually on the headers:因此,您必须在标题上手动设置它:
cookie = response.headers.getlist('Set-Cookie')[0].split(';')[0]
FormRequest.from_response(response,
formdata={"username": "asldkfs", "pass": "slskd"},
headers={'Cookie': cookie}, # <---
meta = {'dont_redirect': True,'handle_httpstatus_list': [302]},
callback=self.redirection)
我认为如果您禁用它,您将无法使用 cookie。
Paulo Romeira的回答是正确的,只是缺少从字节到字符串的解析:
cookie = response.headers.getlist('Set-Cookie')[0].decode("utf-8").split(";")[0].split("=")
You can also use the concepts of creating cookiejars and handling the cookies there (for maintaining sessions in scrapy) using the following strategies documented here in the official scrapy docs: https://doc.scrapy.org/en/latest/topics/downloader-middleware.html#std:reqmeta-cookiejar您还可以使用官方scrapy文档中记录的以下策略来使用创建cookiejars和处理cookies的概念(用于在scrapy中维护会话): https ://doc.scrapy.org/en/latest/topics/downloader -middleware.html#std:reqmeta-cookiejar
Also refer this stackoverflow issue: Scrapy - how to manage cookies/sessions另请参阅此 stackoverflow 问题: Scrapy - 如何管理 cookie/会话
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.