[英]urlopen not getting all the data from web (python)
我正在嘗試從網站下載圖片。 我發現為什么我找不到圖片URL的問題直接在代碼的開頭。
我的問題是urlopen下載的HTML與我在瀏覽器中獲得的HTML不同。
該站點在這里 。 當我在瀏覽器中查看HTML時,可以看到以下部分:
<a href="#" data-trigger="cmg-rotate-big">
<img src="/image/product/eca412b9-9484-4046-8bee-8400fde1d5fe/?width=400" alt="" data-cm-index="0" style="width: 400px; height: 400px; margin-left: 0px; opacity: 1;">
<img src="/image/product/014a128e-fa7b-4817-9d76-7bdf296de8de/?width=400" alt="" data-cm-index="1" style="width: 0px; height: 400px; margin-left: 200px; opacity: 0.5;">
</a>
但是通過代碼
text = urllib2.urlopen(url).read()
soup = BeautifulSoup(text, "html.parser")
print(soup)
同一部分只是
<a data-trigger="cmg-rotate-big" href="#">
<img alt="" data-cm-index="0" src=""/>
<img alt="" data-cm-index="1" src=""/>
</a>
所以我可以提取圖像的SRC,因為它丟失了..請問問題出在哪里?
謝謝!
src href在其中。 無需模擬javascript。
import requests
import bs4
url = 'https://ceskamincovna.cz/stribrna-mince-na-kolech---skoda-felicia-proof-1493-11549-d/'
response = requests.get(url)
soup = bs4.BeautifulSoup(response.text , 'html.parser')
imgs = soup.find_all('img')
for img in imgs:
if '/image/product/' in img['src']:
print (img['src'])
輸出:
/image/product/eca412b9-9484-4046-8bee-8400fde1d5fe/?width=250
/image/product/014a128e-fa7b-4817-9d76-7bdf296de8de/?width=250
/image/product/0ec5b392-0f8a-4013-a448-a1b82578c008/?width=250
/image/product/9bc26462-5f11-4994-be6e-fcde1d97c5f3/?width=250
/image/product/7da1f235-f322-4a57-b0ca-07964f0a7d37/?width=250
/image/product/bd781b17-8482-4a4f-80f3-5fa55b9bc4c1/?width=250
/image/product/f5d4ade9-cac0-4c15-a935-da125b408da1/?width=250
/image/product/f4d6fb41-af72-4510-a70c-0a9893656e93/?width=250
/image/product/6136afe7-7444-42cd-858b-af66ca4ca6de/?width=140
/image/product/a459eb25-dd12-446a-9517-341d128c9571/?width=140
如果您希望寬度= 400:
import requests
import bs4
url = 'https://ceskamincovna.cz/stribrna-mince-na-kolech---skoda-felicia-proof-1493-11549-d/'
response = requests.get(url)
soup = bs4.BeautifulSoup(response.text , 'html.parser')
imgs = soup.find_all('img')
for img in imgs:
if '/image/product/' in img['src']:
print (img['src'].split('?width=')[0] + '?width=400')
輸出:
/image/product/eca412b9-9484-4046-8bee-8400fde1d5fe/?width=400
/image/product/014a128e-fa7b-4817-9d76-7bdf296de8de/?width=400
/image/product/0ec5b392-0f8a-4013-a448-a1b82578c008/?width=400
/image/product/9bc26462-5f11-4994-be6e-fcde1d97c5f3/?width=400
/image/product/7da1f235-f322-4a57-b0ca-07964f0a7d37/?width=400
/image/product/bd781b17-8482-4a4f-80f3-5fa55b9bc4c1/?width=400
/image/product/f5d4ade9-cac0-4c15-a935-da125b408da1/?width=400
/image/product/f4d6fb41-af72-4510-a70c-0a9893656e93/?width=400
/image/product/6136afe7-7444-42cd-858b-af66ca4ca6de/?width=400
/image/product/a459eb25-dd12-446a-9517-341d128c9571/?width=400
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.