Selenium - 下载文件 - click() 仅有时有效

Question

我正在尝试制作一个脚本来从 BlackRock 网站（ishares.com 或 blackrock.com）下载特定的 PDF 文件，但 click() ZC1C425268E68385D1AB5074C17A94F4 通常不起作用。 有时它会这样做 - 每执行 3-5 次左右，它就会设法下载一个文件。

（当我对来自这些网站的所有 PDF 使用类似的脚本时，它在几次执行中也只运行一次，并且每次运行时它总是下载相同的文件，跳过 rest。）

因此，假设我尝试从这些站点下载 KIID/KID PDF 文件：

https://www.ishares.com/uk/individual/en/products/251857/ishares-msci-emerging-markets-ucits-etf-inc-fund?switchLocale=y&siteEntryPassthrough=true
https://www.ishares.com/ch/individual/en/products/251931/ishares-stoxx-europe-600-ucits-etf-de-fund?switchLocale=y&siteEntryPassthrough=true
https://www.blackrock.com/uk/individual/products/251565/ishares-euro-corporate-bond-large-cap-ucits-etf?switchLocale=y&siteEntryPassthrough=true

使用此代码：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from pyvirtualdisplay import Display
import time


def blackrock_getter(url):
    with Display():
        mime_types = "application/pdf,application/vnd.adobe.xfdf,application/vnd.fdf,application/x-pdf,application/vnd.adobe.xdp+xml"
        profile = webdriver.FirefoxProfile()
        profile.set_preference('browser.download.folderList', 2)
        profile.set_preference('browser.download.manager.showWhenStarting', False)
        profile.set_preference('browser.download.dir', '/home/user/kiid_temp')
        profile.set_preference('browser.helperApps.neverAsk.saveToDisk', mime_types)
        profile.set_preference("plugin.disable_full_page_plugin_for_types", mime_types)
        profile.set_preference('pdfjs.disabled', True)
        driver = webdriver.Firefox(firefox_profile=profile)
        driver.get(url)
        try:
            element = WebDriverWait(driver, 20).until(
            EC.element_to_be_clickable((By.XPATH, ("//header[@class='main-header']//a[@class='icon-pdf'][1]"))))
            driver.execute_script("arguments[0].click();", element)
        finally:
            driver.quit()
        time.sleep(3)  # very precise mechanism to wait until the download is complete


def main():
    urls_file = open('urls_list.txt', 'r')  # the URLs I pasted above
    for url in urls_file.readlines():
        if url[-1:] == "\n":
            url = url[:-1]
        if url[0:4] == "http":
            filename = url.split('?')[0]
            filename = filename.split('/')[-1]
            if 'blackrock.com/' in url or 'ishares.com/' in url:
                print(f"Processing {filename}...")
                blackrock_getter(url)


main()

结果是（每隔一段时间）一个文件：kiid-ishares-msci-emerging-markets-ucits-etf-dist-gb-ie00b0m63177-en.pdf。

任何想法如何解决这一问题？

Answer 1

您可以尝试使用pyautogui模块，但您将无法在程序运行时使用您的计算机。

Answer 2

似乎是脚本在文件下载完成之前完成，我的意思是下载不会在 3 秒内竞争。 这是等待 PDF 下载完成的方法。

# method to get the downloaded file name
def getDownLoadedFileName(waitTime):
    driver.execute_script("window.open()")
    # switch to new tab
    driver.switch_to.window(driver.window_handles[-1])
    # navigate to chrome downloads
    driver.get('chrome://downloads')
    # define the endTime
    endTime = time.time()+waitTime
    while True:
        try:
            # get downloaded percentage
            downloadPercentage = driver.execute_script(
                "return document.querySelector('downloads-manager').shadowRoot.querySelector('#downloadsList downloads-item').shadowRoot.querySelector('#progress').value")
            # check if downloadPercentage is 100 (otherwise the script will keep waiting)
            if downloadPercentage == 100:
                # return the file name once the download is completed
                return driver.execute_script("return document.querySelector('downloads-manager').shadowRoot.querySelector('#downloadsList downloads-item').shadowRoot.querySelector('div#content  #file-link').text")
        except:
            pass
        time.sleep(1)
        if time.time() > endTime:
            break