如何在不丢失信息的情况下从谷歌下载图像以及如何使用 Python 使用枕头模块读取它们

Question

I am facing some issue while downloading image from the google search page then saving the same image into disk. I am also facing issue while reading the same image.

Issue 1: (Downloading Image and saving into disk) I have used "requests" module to download the image. once the image is downloaded and if i try to open it, it is showing below error instead of actual image contents(tried all image formats like jpg, png, etc....) "It looks like we don't support this file format"

Note: I have used urllib.requests module as well to download the image, in this case also i am facing same issue.

Below is the code Used:

image_url = "https://www.google.com/imgres?imgurl=https%3A%2F%2Fi.etsystatic.com%2F16576605%2Fr%2Fil%2Fab973a%2F1811762786%2Fil_570xN.1811762786_ni8d.jpg&imgrefurl=https%3A%2F%2Fwww.etsy.com%2Flisting%2F676777770%2F8-styles-wood-acrylic-leather-endless&docid=Knls-viNHmqhZM&tbnid=WF4mlYC28VcOKM%3A&vet=10ahUKEwiB8v3NnezmAhWmzjgGHaWDCtIQMwgrKAAwAA..i&w=570&h=571&itg=1&bih=710&biw=1536&q=676777770&ved=0ahUKEwiB8v3NnezmAhWmzjgGHaWDCtIQMwgrKAAwAA&iact=mrc&uact=8"

with open(temp_file_path, "wb") as fil:
    response = requests.get(image_url, stream=True)
    response.raw.decode_content = True
    shutil.copyfileobj(response.raw, fil)
fil.close()

Issue2: (Open the downloaded image with PIL Module) Next step is to read the downloaded image and i have used "PIL" (pillow) module to do this. But i am facing below issue. "PIL.UnidentifiedImageError: cannot identify image file <_io.BufferedReader name=' path \\1.jpg'>"

Note: If i used manually downloaded images or captured images i am able to read them properly.

Below is the code i have used:

from PIL import Image
img = Image.open(open(temp_file_path, "rb"))

I think this is because of bytes vs string conversion issue but i am not able to figure it out

I am attaching the image which was downloaded by script using requests module for reference.

It would be great someone helps me....

Answer 1

The problem that I see is that that url does not go directly to an image. I tried your code with image_url="https://i.etsystatic.com/16576605/r/il/ab973a/1811762786/il_794xN.1811762786_ni8d.jpg" and everything worked perfectly.

Answer 2

You can download images using urllib.request.urlretrieve(URL, 'your_filename.mp3/jpeg/png/whatever'

import urllib.request

URL = "https://i.etsystatic.com/16576605/r/il/ab973a/1811762786/il_570xN.1811762786_ni8d.jpg"
urllib.request.urlretrieve(URL, "perfect_filename.png")

Sometimes it won't download anything because the request was sent via script (bot), and if you want to parse images from Google images or other search engines, you need to pass user-agent to request headers first, and then download the image, otherwise, the request will be blocked and it will throw an error.

Pass user-agent and download image:

opener=urllib.request.build_opener()
opener.addheaders=[('User-Agent','Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582')]
urllib.request.install_opener(opener)

urllib.request.urlretrieve(URL, 'image_name.jpg')

Code and example in the online IDE that scrapes and download images .

---

Alternatively, you can achieve this by using Google Images API from SerpApi. It's a paid API with a free plan.

The difference is that you don't have to deal with scraping data from the <script> tags or figuring out how to bypass blocks from Google or other search engines since it's already done for the end-user.

Code to integrate:

from serpapi import GoogleSearch
import os

params = {
  "api_key": os.getenv("API_KEY"),
  "engine": "google",
  "q": "pexels cat",
  "tbm": "isch"
}

search = GoogleSearch(params)
results = search.get_dict()

for index, image in enumerate(results['images_results']):

    print(f'Downloading {index} image...')
    
    opener=urllib.request.build_opener()
    opener.addheaders=[('User-Agent','Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19582')]
    urllib.request.install_opener(opener)

    urllib.request.urlretrieve(image['original'], f'SerpApi_Images/original_size_img_{index}.jpg')

Disclaimer, I work for SerpApi.