web刮HTML表

Question

How could i scrape opening odds on this site https://www.betexplorer.com/soccer/russia/premier-league/arsenal-tula-ufa/IwvO3Q5T/ ?

try:
    driver.find_element_by_xpath("//td[a[.='bet365']]/following-sibling::td[span]")
except NoSuchElementException:
    homeodd = 'no bet365 odd'
    drawodd = 'no bet365 odd'
    awayodd = 'no bet365 odd'
else:
    homeodd = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//td[a[.="bet365"]]/following-sibling::td[span][1]'))).get_attribute("data-opening-odd")
    drawodd = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//td[a[.="bet365"]]/following-sibling::td[span][2]'))).get_attribute("data-opening-odd")
    awayodd = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, '//td[a[.="bet365"]]/following-sibling::td[span][3]'))).get_attribute("data-opening-odd")

Answer 1

Proposal the opening home odds:

WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).click()
oid = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).get_attribute("@data-oid")
bid = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "(//td[a[.='bet365']]/following-sibling::td[@data-odd])[1]"))).get_attribute("@data-bid")
var = oid+'-'+bid
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[contains(@id,'%s')]/tr[last()]/td[@class='bold']" % var))).text

First we click on the td element containing the final odd. Then we grab 2 attribute values ( data-oid and data bid ) from this td element. We concat these 2 values into a variable. We use this variable in our last XPath expression to locate the td element which contains the opening odd.

For draw odds and away odds, use the following XPath:

(//td[a[.='bet365']]/following-sibling::td[@data-odd])[2]
(//td[a[.='bet365']]/following-sibling::td[@data-odd])[3]

Add exceptions in case there's no opening odds to get (no opening draw odds for bet365 in your sample webpage).

Imports:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC