简体   繁体   中英

How do you scrape javascript using selenium & BS3?

I'm having trouble with scraping javascript.
I want to scrape some numbers from this ESPN webpage but I had the look at the HTML and have no idea how. It seems like I've tried everything that I know but I can't. So how, for example, do I scrape the 435 or 677 located under meters run?
This is the current code I have :

url = "http://www.espn.co.uk/rugby/matchstats?gameId=291168&league=244293"
browser = webdriver.Firefox()
browser.get(url)

soup = BeautifulSoup(browser.page_source, "html.parser")
teams_spans = soup.find_all("span", { "class" : "long-name" })
home_team = teams_spans[0].text
away_team = teams_spans[1].text  

Want to scrape the data but have no clue how.

I used Selenium for parsing the elements instead of using BeautifulSoup.

Code:

from selenium import webdriver

driver = webdriver.Chrome()
driver.get('http://www.espn.co.uk/rugby/matchstats?gameId=291168&league=244293')

spans = driver.find_elements_by_class_name('chartValue')
for span in spans[0:2]:
    print(span.text)

Output:

435
677

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM