[英]BeautifulSoup doesn't find tables on webpage
I'm trying to get the data from the 1st table on a website.我正在尝试从网站上的第一个表中获取数据。 I've looked on here for similar problems and tried a number of the given solutions but can't seem to find the table and ultimately the data in the table.我在这里查看了类似的问题并尝试了许多给定的解决方案,但似乎无法找到表格并最终找到表格中的数据。
I've tried:我试过了:
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Chrome('C:\\folder\\chromedriver.exe')
url = 'https://docs.microsoft.com/en-us/windows/release-information/'
driver.get(url)
tbla = driver.find_element_by_name('table') #attempt using by element name
tblb = driver.find_element_by_class_name('cells-centered') #attempt using by class name
tblc = driver.find_element_by_xpath('//*[@id="winrelinfo_container"]/table[1]') #attempt by using xpath
and tried using beautiful soup并尝试使用美丽的汤
html = driver.page_source
soup = BeautifulSoup(html,'html.parser')
table = soup.find("table", {"class": "cells-centered"})
print(len(table))
Any help is much appreciated.任何帮助深表感谢。
Table is present inside an iframe
you need to switch iframe
first to access the table
.表存在于iframe
您需要先切换iframe
才能访问该table
。
Induce WebDriverWait()
and wait for frame_to_be_available_and_switch_to_it
() and following locator. Induce WebDriverWait()
并等待frame_to_be_available_and_switch_to_it
() 和以下定位器。
Induce WebDriverWait()
and wait for visibility_of_element_located
() and following locator. Induce WebDriverWait()
并等待visibility_of_element_located
() 和后面的定位器。
driver.get("https://docs.microsoft.com/en-us/windows/release-information/")
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"winrelinfo_iframe")))
table=WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.CSS_SELECTOR,"table.cells-centered")))
You need to import below libraries.您需要导入以下库。
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Or you use below code with xpath
.或者您将以下代码与xpath
。
driver.get("https://docs.microsoft.com/en-us/windows/release-information/")
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"winrelinfo_iframe")))
table=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.XPATH,'//*[@id="winrelinfo_container"]/table[1]')))
You can import further your table data to pandas dataframe and then export to csv file.You need to import pandas.您可以将表数据进一步导入到熊猫数据框,然后导出到 csv 文件。您需要导入熊猫。
driver.get("https://docs.microsoft.com/en-us/windows/release-information/")
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"winrelinfo_iframe")))
table=WebDriverWait(driver,10).until(EC.presence_of_element_located((By.XPATH,'//*[@id="winrelinfo_container"]/table[1]'))).get_attribute('outerHTML')
df=pd.read_html(str(table))[0]
print(df)
df.to_csv("path/to/csv")
Import pandas: pip install pandas
导入熊猫: pip install pandas
Then add the below library然后添加下面的库
import pandas as pd
The table is located inside <iframe>
, so BeautifulSoup
doesn't see it inside original page:该表位于<iframe>
,因此BeautifulSoup
在原始页面内看不到它:
import requests
from bs4 import BeautifulSoup
url = 'https://docs.microsoft.com/en-us/windows/release-information/'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')
soup = BeautifulSoup(requests.get(soup.select_one('iframe')['src']).content, 'html.parser')
for row in soup.select('table tr'):
print(row.get_text(strip=True, separator='\t'))
Prints:印刷:
Version Servicing option Availability date OS build Latest revision date End of service: Home, Pro, Pro Education, Pro for Workstations and IoT Core End of service: Enterprise, Education and IoT Enterprise
2004 Semi-Annual Channel 2020-05-27 19041.546 2020-10-01 2021-12-14 2021-12-14 Microsoft recommends
1909 Semi-Annual Channel 2019-11-12 18363.1110 2020-09-16 2021-05-11 2022-05-10
1903 Semi-Annual Channel 2019-05-21 18362.1110 2020-09-16 2020-12-08 2020-12-08
1809 Semi-Annual Channel 2019-03-28 17763.1490 2020-09-16 2020-11-10 2021-05-11
1809 Semi-Annual Channel (Targeted) 2018-11-13 17763.1490 2020-09-16 2020-11-10 2021-05-11
1803 Semi-Annual Channel 2018-07-10 17134.1726 2020-09-08 End of service 2021-05-11
...and so on.
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.