简体   繁体   中英

Scraping web page with Requests Python

I would to scrape this page but I cant find table with BeautifulSoup after requests

Code

headers = {"Referer": "https://www.atptour.com/en/scores/results-archive",
            'User-Agent': 'my-user-agent'
        }
url = 'https://www.atptour.com/en/scores/results-archive?year=2016'
page = requests.get(url, headers=headers)
print(page)
soup = BeautifulSoup(page.text, 'html.parser')
table = soup.find('table', class_="results-archive-table mega-table")
print(table)

Output <Response [403]> None

Look at the response :

print(page)
<Response [403]>

Maybe you have to add some headers to your request.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM