用 Selenium 抓取一個表並打印出來

Question

我想從網站上抓取所有表格。 到達表格需要自動化，因此您可以考慮這一點。 我的研究嘗試如下：

from selenium.webdriver import Firefox
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains
import time

driver = Firefox(executable_path='/Users/.../PycharmProjects/Sportwinner/geckodriver')
driver.get("https://bskv.sportwinner.de/")
element = driver.find_element(By.ID, "id-button-einstellungen")
actions = ActionChains(driver)
actions.move_to_element(element).perform()
driver.find_element(By.ID, "id-button-einstellungen").click()
element = driver.find_element(By.CSS_SELECTOR, "body")
actions = ActionChains(driver)
actions.move_to_element(element).perform()
driver.find_element(By.ID, "id-klub-name").click()
driver.find_element(By.ID, "id-klub-name").send_keys("Dreieck Schweinfurt")
driver.find_element(By.ID, "id-button-einstellungen-option-ok").click()
time.sleep(1)
driver.find_element(By.ID, "id-dropdown-liga").click()
driver.find_element(By.LINK_TEXT, "Letzte Spielwoche").click()

tableContent = driver.find_elements_by_css_selector("id-table-spiel tr")
for row in tableContent:
    print(row.text)

因為我幾個小時前才聽說 Selenium，所以我完全是個菜鳥。 我不知道這是否有效，因為我沒有看到任何 output。是否有人能夠幫助我進行嘗試（我猜這是不正確的）以及我如何才能看到結果？ 我正在使用 PyCharm 進行編譯。

Answer 1

執行速度如此之快以至於無法從表中提取詳細信息。

您需要應用Implicit wait或Explicit waits ，以便顯示表數據並能夠提取詳細信息。

# Imports Required
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

...
driver = webdriver.Chrome(executable_path="chromedriver.exe") # Have tried in Chrome
driver.implicitly_wait(20)

# Or apply Explicit wait like below.
wait = WebDriverWait(driver,30)
wait.until(EC.presence_of_element_located((By.XPATH,"//table[@id='id-table-spiel']//tbody/tr")))

tableContent = driver.find_elements_by_xpath("//table[@id='id-table-spiel']//tbody/tr//div")
for row in tableContent:
    print(row.get_attribute("innerText")) # row.text works too.

你可以這樣嘗試：

tableContent = driver.find_elements_by_xpath("//table[@id='id-table-spiel']//tbody/tr//a")

for i in range(len(tableContent)):
    tableContent[i].click() # Clicks on the "+" icon
    innerrows = driver.find_elements_by_xpath("//tr[@class='detail-view'][{}]//tr".format(i+1)) #Find the rows inside the 1st and 2nd row.
    for inrow in innerrows:
        elemnets = inrow.find_elements_by_xpath(".//div") # Data are in "div" tags
        data = [] #Collect each row in a list
        for j in elemnets:
            data.append(j.text)
        print(data)

['', '', '1', '', '2', '', '3', '', '4', '', 'Kegel', '', 'SP', '', 'MP', '', '', '', 'MP', '', 'SP', '', 'Kegel', '', '4', '', '3', '', '2', '', '1', '', '', '']
['Krug, Tobias', '141', '141', '136', '86', '141', '152', '124', '131', 'Brandl, Gerald']
['Keller, Ralf', '148', '135', '139', '130', '140', '111', '154', '145', 'Haschke, Jens']

Answer 2

到達所需頁面后， Letzte Spielwoche" in the drop down menu "Eine Liga auswählen" to see the tables

您可以使用此代碼：

wait = WebDriverWait(driver, 30)
table = wait.until(EC.visibility_of_element_located((By.ID, "id-table-spiel")))
size_of_table = driver.find_elements(By.XPATH, "//table[@id='id-table-spiel']//descendant::tr")
j = 1
for i in range(len(size_of_table)):
    element = driver.find_elements(By.XPATH, f"(//table[@id='id-table-spiel']//descendant::tr)[{j}]")
    driver.execute_script("arguments[0].scrollIntoView(true);", element)
    print(element.get_attribute('innerText'))
    j = j + 1

進口：

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC