[英]BeautifulSoup webscraper issue: can't find certain divs/tables
[英]beautifulsoup webscraper problem: can't find tables on webpage
我想从表中获取该网站使用此代码:
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
my_url = 'https://www.flashscore.pl/pilka-nozna/'
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()
page_soup = soup(page_html, "html.parser")
containers = page_soup.find_all('table', {'class': 'soccer'})
print(len(containers))
但是,当我尝试检查通过print(len(containers))
得到多少张表时,我得到0。有什么解决办法?
编辑:
页面可能是动态的。 您可以使用requests-html ,它允许您在拉出html之前让页面呈现,也可以像我在此处使用Selenium一样。
这产生了表class =“ soccer”的42个元素
import bs4
from selenium import webdriver
url = 'https://www.flashscore.pl/pilka-nozna/'
browser = webdriver.Chrome('C:\chromedriver_win32\chromedriver.exe')
browser.get(url)
html = browser.page_source
soup = bs4.BeautifulSoup(html,'html.parser')
containers = soup.find_all('table', {'class': 'soccer'})
browser.close()
In [11]: print(len(containers))
42
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.