Python Selenium 使用 ChromeDriver 下载图像（jpeg、png）或 PDF

Question

I have a Selenium script in Python (using ChromeDriver on Windows) that fetches the download links of various attachments(of different file types) from a page and then opens these links to download the attachments. This works fine for the file types which ChromeDriver can't preview as they get downloaded by default. But images(JPEG, PNG) and PDFs are previewed by default and hence aren't automatically downloaded.

The ChromeDriver options I am currently using (work for non preview-able files) :

chrome_options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : 'custom_download_dir'}
chrome_options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome("./chromedriver.exe", chrome_options=chrome_options)

This downloads the files to 'custom_download_dir', no issues. But the preview-able files are just previewed in the ChromeDriver instance and not downloaded.

Are there any ChromeDriver Settings that can disable this preview behavior and directly download all files irrespective of the extensions?

If not, can this be done using Firefox for instance?

Answer 1

Instead of relying in specific browser / driver options I would implement a more generic solution using the image url to perform the download.

You can get the image URL using similar code:

driver.find_element_by_id("your-image-id").get_attribute("src")

And then I would download the image using, for example, urllib.

Here's some pseudo-code for Python2:

import urllib

url = driver.find_element_by_id("your-image-id").get_attribute("src")
urllib.urlretrieve(url, "local-filename.jpg")

Here's the same for Python3:

import urllib.request

url = driver.find_element_by_id("your-image-id").get_attribute("src")
urllib.request.urlretrieve(url, "local-filename.jpg")

Edit after the comment, just another example about how to download a file once you know its URL:

import requests
from PIL import Image
from io import StringIO

image_name = 'image.jpg'
url = 'http://example.com/image.jpg'

r = requests.get(url)

i = Image.open(StringIO(r.content))
i.save(image_name)

Answer 2

With selenium-wire library, it is possible to download images via ChromeDriver .

I have defined the following function to parse each request and save the request body to a file when necessary.

import os
from mimetypes import guess_extension
from seleniumwire import webdriver

def download_assets(requests, asset_dir="temp", default_fname="untitled", exts=[".png", ".jpeg", ".jpg", ".svg", ".gif", ".pdf", ".ico"]):
    asset_list = {}
    for req_idx, request in enumerate(requests):
        # request.headers
        # request.response.body is the raw response body in bytes
        ext = guess_extension(request.response.headers['Content-Type'].split(';')[0].strip())
        if ext is None or ext not in exts:
            #Don't know the file extention, or not in the whitelist
            continue

        # Construct a filename
        fname = os.path.basename(request.url.split('?')[0])
        fname = "".join(x for x in fname if (x.isalnum() or x in "._- "))
        if fname == "":
            fname = f"{default_fname}_{req_idx}"
        if not fname.endswith(ext):
            fname = f"{fname}{ext}"
        fpath = os.path.join(asset_dir, fname)

        # Save the file
        print(f"{request.url} -> {fpath}")
        asset_list[fpath] = request.url
        with open(fpath, "wb") as file:
            file.write(request.response.body)
    return asset_list

Let's download some images from Google homepage to temp folder.

# Create a new instance of the Chrome/Firefox driver
driver = webdriver.Chrome()

# Go to the Google home page
driver.get('https://www.google.com')

# Download content to temp folder
asset_dir = "temp"
os.makedirs(asset_dir, exist_ok=True)
download_assets(driver.requests, asset_dir=asset_dir)

driver.close()

Note that the function can be improved such that the directory structure can be kept as well.

Answer 3

Here is another simple way, but @Pitto's answer above is slightly more succinct.

import requests

webelement_img = ff.find_element(By.XPATH, '//img')
url = webelement_img.get_attribute('src') or 'https://someimages.com/path-to-image.jpg'
data = requests.get(url).content
local_filename = 'filename_on_your_computer.jpg'

with open (local_filename, 'wb') as f:
    f.write(data)