如何通过存储在 pandas dataframe 的直接链接下载所有文件

Question

I have following pandas dataframe .

filename   Direct url
file1.pdf  www.abc.com/file1.pdf, www.abc.com/file3.pdf
file2.pdf  www.abc.com/file2.pdf

I want to download directly download these files using Python, Selenium on Firefox browser I have written following code

dl_dir = "path/to/dl/folder"

ff_profile = webdriver.FirefoxProfile()

ff_profile.set_preference("browser.download.folderList", 2)
ff_profile.set_preference("browser.download.manager.showWhenStarting", False)
ff_profile.set_preference("browser.download.dir", dl_dir)
ff_profile.set_preference('browser.helperApps.neverAsk.saveToDisk', "text/plain, application/vnd.ms-excel, text/csv, text/comma-separated-values, application/pf")

driver = webdriver.Firefox(ff_profile)

for index,row df.iterrows():
    dir_urls=df[df["filename"]==row["filename"]]["Direct url"].iat[0]
    url_lst=dir_urls.split(",")
    for i in url_lst:
        driver.get(i)

But I am getting timeout exception saying Timeout loading page , as I'm not trying to open any webpage here, I'm just trying auto download the file. Currently, I'm only able to download 1 file as it gives timeout exception after that. How can I circumvent this and download all the files?

Answer 1

I would suggest that you increase timeout delay:

driver.set_page_load_timeout(30)  # seconds

And you could use a try/except statement:

try:
    for i in url_lst:
        driver.get(i)
except [exception]:
    continue