简体   繁体   中英

Failing to scrape a table with beautifulsoup

I am trying to scrape a table on a webpage, which I want to turn into a pandas DataFrame. The page I am trying to scrape requires authentication but I have managed to pass it using the request package. Next, I want to scrape the table and I found using the dev tools in Chrome. I have copied the selector and passed it to the soup selector() method. However, when I print it out, it returns an empty string. I have tried several different approaches, all of which fail to give me the table that I so desperately want. What am I doing wrong? This is my code:

import requests
from bs4 import BeautifulSoup as bs


cookies = {
#some information
}

headers = {
 #some information
}

params = (
    #some information
)
response = requests.get('http://www.hctiming.com/myphp/resources/login/browse_results.php?live_action=yes&smartphone_action=no', headers=headers, cookies=cookies, verify=False)

soup = bs(response.content, features="lxml")

test = soup.select("#fis_result_content0 > div.row.racers-list-tab.ranking > div > div > table")
print(test)

And this is a screenshot of my dev.tools in chrome, just to give you some idea about where my table sits:

在此处输入图片说明

Instead of this:

test = soup.select("#fis_result_content0 > div.row.racers-list-tab.ranking > div > div > table")

Try this :

soup.find("table",class_="ranking_table")

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM