Python web scraping javascript table using selenium

Question

Hello to all this is my first post hope you are good!

I try to extract the content of the table located on this site: https://pamestoixima.opap.gr/ . I want to extract the table as is. I have tried several ways such as beautifulSoup, pandas, and selenium but with no success: The latest code I have tried is this:

from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Chrome( executable_path=r'C:path to /chromedriver.exe')
driver.get('https://pamestoixima.opap.gr/')

soup = BeautifulSoup(driver.page_source,'lxml')
author_element = soup.find("table", class_="results-table")

print(author_element.text)
driver.quit()

The error message I get is this:

USB: usb_device_handle_win.cc:1049 Failed to read descriptor from node connection:

Thank you in advance for any help!

Answer 1

Do you specifically want to use Selenium? I haven't looked into detail but I think you can find all the data in this file: https://api.opap.gr/sb/sport/soccer/coupon?locale=el&onlyLive=false&marketIds=1%2C2%2C31%2C21%2C18%2C18%2C18%2C14&fromDate=2022-05-03&toDate=2022-05-03

In many cases you can use the Developer Tools in your browser to find a direct link to the data instead of scraping the page, which is usually much more difficult.