简体   繁体   中英

Javascript-based web logon for page scraping

I'm trying to write a scraping engine for sbrodds.com. Unfortunately I need to login to get the right data. I looked into mechanize and selenium but I am sadly very poor at python and web scraping and am not understanding how to use these.

Here is a screenshot of the Chrome inspect elements page for the login boxes: Chrome检查结果

Can someone please provide as suggestion as to what technology I should use to perform a login to this site inside Python code? The goal is to eventually load the logged-in page's data into BeautifulSoup.

usually just requests is enough ... why do you need javascript based?

import requests
r = requests.Session()
r.post("http://sbrodds.com/login",{"username":"bob","password":"sagat"})

content = r.get("http://sbrodds.com/some_page").text

something like that at least ... (you might need a csrf_token or something as well)

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM