Select elements from two different dropdown menus using python selenium

Question

I am trying to scrape data from a dynamic table. To do this, I need to select elements from two different dropdown menus iteratively. First, I want to select the first element of dropdown_secciones, then I want to select the fist element of dropdown_circuitos and finally click the show button (mostrar_click.click()). This would be the first iteration. The second iteration should select the second element of dropdown_circuitos and then click the show button again. No new elements from the first dropdown (dropdown_secciones) should be selected until all elements are selected from dropdown_circuitos.

This is the code I have so far which isn´t working properly:

driver = webdriver.Chrome('/Users/Administrador/Documents/chromedriver')
main_url = 'https://www.justiciacordoba.gob.ar/Estatico/JEL/Escrutinios/ReportesEleccion20190512/default.html'
driver.get(main_url)

driver.switch_to.frame("topFrame")

dropdown_secciones = driver.find_element_by_xpath('./html/body/table/tbody/tr/td/table/tbody/tr[2]/td/table/tbody/tr[1]/td[2]/select')
select_box_secciones = Select(dropdown_secciones)
options_secciones = select_box_secciones.options

dropdown_circuitos = driver.find_element_by_xpath('//*[@id="cmbCircuitos"]')
select_box_circuitos = Select(dropdown_circuitos)
options_circuitos = select_box_circuitos.options

mostrar_click = driver.find_element_by_xpath('/html/body/table/tbody/tr/td/table/tbody/tr[3]/td/div/input[1]')

for index in range(0, len(options_secciones) - 1):
    select_box_secciones.select_by_index(index)
    mostrar_click.click()

    for i in range(0, len(options_circuitos) - 1):

        select_box_circuitos.select_by_index(i)
        circuitos.append(select_box_circuitos.get_attribute('innerHtml'))
        mostrar_click.click()

I am quite new with selenium but I guess I should some how break the first iteration when an element is found and then "jump" to the second interation. Anyway, any ideas on how to improve the code??

Answer 1

I think I've come up with something that should help. Here are some notes:

• The second dropdown behaviour changes as you work with the first, so I've moved that stuff in the loop itself
• Some frame switches are needed so I've just added some functions to make it easier (switch_to_top and switch_to_main)
• The index in the for loops should start at 1 as 0 will select the default value
• When the table loads, I've just added a wait to cater for this
• I've used find by id in some places where you've used xpath as I believe it's better practice

One line I wasn't sure about was circuitos.append(select_box_circuitos.get_attribute('innerHtml')) so I've left that, I don't know what the intention here is but I don't think that line will work. You can comment this out initially to check the rest works.

Also ensure to add the imports for the wait:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome('/Users/Administrador/Documents/chromedriver')

def switch_to_top():
    driver.switch_to.default_content()
    driver.switch_to.frame("topFrame")

def switch_to_main():
    driver.switch_to.default_content()
    driver.switch_to.frame("mainFrame")

main_url = 'https://www.justiciacordoba.gob.ar/Estatico/JEL/Escrutinios/ReportesEleccion20190512/default.html'
driver.get(main_url)

switch_to_top()

dropdown_secciones = driver.find_element_by_id('cmbSecciones')
select_box_secciones = Select(dropdown_secciones)
options_secciones = select_box_secciones.options

mostrar_click = driver.find_element_by_id('cmdMostrar')

for index in range(1, len(options_secciones)):
    if (index > 1):
        switch_to_top()
    select_box_secciones.select_by_index(index)

    dropdown_circuitos = driver.find_element_by_id('cmbCircuitos')
    select_box_circuitos = Select(dropdown_circuitos)
    items_circuitos = select_box_circuitos.options

    for i in range(1, len(items_circuitos)):
        if (i > 1):
            switch_to_top()
        select_box_circuitos.select_by_index(i)
        circuitos.append(select_box_circuitos.get_attribute('innerHtml'))
        mostrar_click.click()
        switch_to_main()
        WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "body>table")))