[英]Need help logging in to a website using python
I need to crawl a few websites for a university project and I have reached a dead end for a site that requires log-in. 我需要抓取一些用于大学项目的网站,但是对于需要登录的网站来说,我已经陷入僵局。 I am using the urllib,urllib2,cookielib modules in Python to log in. It does not work for http://www.cafemom.com .
我使用Python中的urllib,urllib2,cookielib模块登录。不适用于http://www.cafemom.com 。 The http reponse that I receive gets saved in a .txt file and corresponds to the 'unsuccessful log-in' page.
我收到的http响应保存在一个.txt文件中,并且对应于“登录失败”页面。
I also tried using the package "twill" for this purpose, which didn't work out for me either. 我也尝试使用软件包“ twill”来实现此目的,这对我也不可行。 Can anyone suggest what I should do?
谁能建议我该怎么办?
Below is the main login() method that I used for this purpose. 下面是我用于此目的的主要login()方法。
def urlopen(req):
try:
r = urllib2.urlopen(req)
except IOError, e:
if hasattr(e, 'code'):
print 'The server couldn\'t fulfill the request.'
print 'Error code: ', e.code
elif hasattr(e, 'reason'):
print 'We failed to reach a server.'
print 'Reason: ', e.reason
raise
return r
class Cafemom:
"""Communication with Cafemom"""
def __init__(self, cookieFile = 'cookie.jar', debug = 0):
self.cookieFile = cookieFile
self.debug = debug
self.loggedIn = 0
self.uid = ''
self.email = ''
self.passwd = ''
self.cj = cookielib.LWPCookieJar()
if os.path.isfile(cookieFile):
self.cj.load(cookieFile)
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(self.cj))
urllib2.install_opener(opener)
def __del__(self):
self.cj.save(self.cookieFile)
def login(self, email, password):
"""Logging in Cafemom"""
self.email = email
self.passwd = password
url='http://www.cafemom.com/lohin.php?'
cnt='http://www.cafemom.com'
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
body = {'identifier': email, 'password': password }
if self.debug == 1:
print "Logging in..."
req = urllib2.Request(url, urllib.urlencode(body), headers)
print urllib.urlencode(body)
#print req.group()
handle = urlopen(req)
h = handle.read()
f = open("responseCafemom.txt","w")
f.write(f)
f.close()
I also tried using this code and was unsuccessful 我也尝试使用此代码,但未成功
import urllib, urllib2, cookielib
username = myusername
password = mypassword
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
login_data = urllib.urlencode({'identifier' : username, 'password' : password})
opener.open('http://www.cafemom.com/login.php', login_data)
resp = opener.open('http://www.cafemom.com')
print resp.read()
I'm not sure if this is exactly what you need, but it's worth a try.The excellent requests module for python supports both cookies, and HTTP basic auth. 我不确定这是否正是您所需要的,但是值得一试.python的出色请求模块支持cookie和HTTP基本身份验证。
These examples are straight from its documentation. 这些示例直接来自其文档。
Here is the basic auth example 这是基本的身份验证示例
payload = {'identifer': email, 'password': password}
r = requests.post("http://www.cafemom.com/login.php?", data=payload)
Here is how to pass cookies previously saved(which you could access from a previous request with "r.cookies".Cookie stores are just dictionaries. 这是传递先前保存的Cookie的方法(您可以使用“ r.cookies”从先前的请求中访问。Cookie商店只是字典。
r = requests.get(url, cookies=cookies)
Here is how to read the response of your request 这是如何阅读您的请求的响应
f = open("responseCafemom.txt","w")
f.write(r.text)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.