Selenium Python - download button .click() no longer working

Question

I have the following code which was working until recently - the aim is to download some historic data from the site which is shown in the code:

import os
import time
import pandas as pd
from datetime import datetime
from selenium import webdriver

# Set option variables for using ChromeDriver

options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable_dev-shm-usage')

options.add_experimental_option("prefs", {
  "download.default_directory": r"\\filepath\foldernamefordownload",
  "download.prompt_for_download": False,
  "download.directory_upgrade": True,
  "safebrowsing.enabled": True
})

# Set up ChromeDriver and open path

wd = webdriver.Chrome(r'filepath\chromedriver.exe', options=options)

solar_path = 'https://www.solar.sheffield.ac.uk/pvlive/'
wd.get(solar_path)
time.sleep(10)

# Click on data download ribbon

data_dwnload1 = wd.find_element_by_id('dataDownloadPanelTitle')
data_dwnload1.click()
time.sleep(5)

# Download data

data_dwnload2 = wd.find_element_by_id('download-csv-button')
data_dwnload2.click()
time.sleep(30)

wd.close()
del solar_path

The problem is that this no longer works - nothing downloads. What am I missing here? The code still runs (there is code after this section which only breaks when it tries to look for the downloaded file which no longer exists).

Any help would be much appreciated.

Answer 1

Simplified a bit bu using explicit waits and it is working. Please check. The time.sleep(30) seems optional. You can wait for a while and then close, but it's your choice if you want to wait for 30 sec.

wd = webdriver.Chrome(r'filepath\chromedriver.exe', options=options)
solar_path = 'https://www.solar.sheffield.ac.uk/pvlive/'
wd.get(solar_path)
time.sleep(10)
# Click on data download ribbon
WebDriverWait(wd, 20).until(EC.visibility_of_element_located((By.ID, 'dataDownloadPanelTitle'))).click()
# Download data
WebDriverWait(wd, 20).until(EC.visibility_of_element_located((By.ID, 'download-csv-button'))).click()
time.sleep(30)

wd.close()

downloaded file snapshot: click here for file snapshot