Failed - Download Error when downloading in Selenium headless Chrome

Question

I am trying to download files in Chrome with Selenium. I discovered that headless Chrome does not allow file downloads by default, and applied a workaround . However, implementing the workaround caused some files to produce a Failed - Download Error in Chrome.

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': abs_path}}
driver.execute('send_command', params)

Here is what my code looks like:

chrome_options = webdriver.ChromeOptions()
prefs = {
    "download.prompt_for_download": False,  # allow automatic downloads
    "plugins.always_open_pdf_externally": True,  # allow download of pdf instead of open in plugin
    "download.default_directory": path,
    "safebrowsing.enabled": False  # allow download of .msi, .exe files, etc.
}
chrome_options.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=chrome_options)

driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': path}}
driver.execute('send_command', params)

for url in file_urls:  # file_urls here is a list of download links
    driver.get(url)

After searching for common reasons for Download error , things I have ruled out are:

1. Incorrect download path: some files with the same download path can be downloaded, but others can
2. File path too long: some files that can be downloaded have longer paths than those with errors

After removing the workaround, all files are able to download as per normal, but I would then be unable to download in headless mode. Any suggestions would be helpful.

Additional information:
ChromeDriver version: 2.40.565498
Chrome version: 67.0.3396.87

Answer 1

you can try disabling security , maybe it will work

what i would recommend is not to run in headless mode, we run selenium in Linux servers too, and we choose to use separate selenium docker containers (webserver) and create remote drivers instead of local ones:

check this page for more information

after you install docker you can launch selenium using this command

docker run -d -p 4444:4444 -v /dev/shm:/dev/shm --network host selenium/standalone-chrome:3.141.59-neon

make sure to have --network host so that the 4444 port can be accessed from localhost . after that, you can simply create a remote driver by doing this

from selenium import webdriver

driver = webdriver.Remote(command_executor = exec_path or 'http://localhost:4444/wd/hub'))

this way you won't have to worry about anything from selenium, and focus on your project