Python 网页抓取：使用 urllib 时图像不完整

Question

I am trying to retrieve an image using Python and BeautifulSoup. I managed to get the full url of the image but when I use urllib.urlretrieve(imagelink, filename) , it retrieves the image but the image is incomplete, only 3.2kb.

The real images (im getting a lot of images) average around 800kb. It iterates through and downloads all the images but none of them are viewable and are all the same filesize. The full image urls work fine when opened in the browser though.

Any idea what could cause such an issue? I don't think showing my code would help but here is the section where I am getting the url:

print imagelink
filename = imagelink.split('/')[-1]
time.sleep(5)
urllib.urlretrieve(imagelink, filename)
time.sleep(5)

Answer 1

宏杰李, requests is a wrapper for urllib. As it is also a wrapper for sockets -))

With urllib2 the same result can be achieved like this.

>>> import urllib2
>>> r = urllib2.urlopen('https://i.stack.imgur.com/tkGEv.jpg?s=328&g=1')
>>> with open("/home/ziya/Pictures/so_image.jpg", "wb") as img:
...     img.write(r.read())

Answer 2

You should try requests :

import requests
url = 'https://i.stack.imgur.com/tkGEv.jpg?s=328&g=1'
r = requests.get(url)
with open('tkGEv.jpg', 'wb') as fd:
    for chunk in r.iter_content(chunk_size=128):
        fd.write(chunk)