How to interact with a non-select dropdown with selenium python

Question

I am currently learning the use of selenium with python, and tried to collect some data. I have been struggling for the last couple days with clicking on a dropdown not accessible by Select method. I looked at A LOT of questions on SOF, blogs, tutorials... and could not find the answer to my problem.

The dropdown is accessible to this website <"https://en.volleyballworld.com/volleyball/competitions/olympics-2020/schedule/11349/">, then by clicking on the "Box Score" tab. Just below the teams flags, you'll see the dropdown with "ALL SETS" writing in it.

I would like to access the data from "SET 1", "SET 2", "SET 3". My guess would be to click on the dropdown, then click on "SET 1" and so on. But I couldn't make the code work to click on the dropdown.

Below is my code:

PATH = "C:\Program Files\chromedriver.exe"
driver = webdriver.Chrome(PATH)

driver.get("https://en.volleyballworld.com/volleyball/competitions/olympics-2020/schedule/11349/")

#implicit wait to be sure the elements we want are loaded when we try accessing them
driver.implicitly_wait(5)
actions = ActionChains(driver)


#clicking on button
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, "tab-title_boxscore")))
element.click() #mimic clicking on the clickable element
dropdown = WebDriverWait(driver, 10).until(EC.element_to_be_clickable(
    (By.LINK_TEXT, "ALL SETS"))).click()
first_set = WebDriverWait(driver, 10).until(EC.element_to_be_clickable(
    (By.LINK_TEXT, "SET 1"))).click()

Many thanks for your time and answer !

Answer 1

To click() on the element with text as ALL SETS and then to click on the element with text as SET 1 instead of presence_of_element_located() you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following Locator Strategies :

Code Block:

driver.get("https://en.volleyballworld.com/volleyball/competitions/olympics-2020/schedule/11349/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span.tab-title_boxscore"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "span.tab-title_all"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='All Sets']//following::li[1]/a/span"))).click()

Browser Snapshot: