簡體   English   中英

我正在使用Python 3.7和BS4進行網絡抓取,這是我無法解決的問題,希望有人知道如何解決此問題

[英]I'm using Python 3.7 an BS4 for web scraping, there is a problem I couldn't solve, hope someone knows how to fix this

我想從源頁面獲取產品信息,我想要的數據在HTML標記中 ,但標記中還有另一個標記,因此當我將數據保存到本地存儲時,它看起來非常糟糕。 我希望有人知道如何解決此問題。

這是我的代碼:

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup

my_url = 'https://list.jd.com/list.html? 
cat=9987,653,655&ev=exbrand_15127&page=1'

#opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

#html parsing
page_soup = soup(page_html, "html.parser")

filename = "params.csv"
f = open(filename,"w")
#grabs each product
li_containers =  page_soup.findAll("li",{"class":"gl-item"})
for i in range(0,len(li_containers)):
   p_name_div = li_containers[i].find("div",{"class":"p-name"})
   p_name = p_name_div.a.em.text.strip()
   print(p_name)
   f.write(p_name)
f.close()

有一些截圖。

我希望它像這樣:

我希望它像這樣:

但最終看起來像這樣:

但它看起來像這樣:

沒有跨度標簽

帶跨度標簽

嘗試這個

my_url = 'https://list.jd.com/list.html? 
cat=9987,653,655&ev=exbrand_15127&page=1'

#opening up connection, grabbing the page
uClient = uReq(my_url)
page_html = uClient.read()
uClient.close()

#html parsing
page_soup = soup(page_html, "html.parser")

filename = "params.csv"
f = open(filename,"w")
#grabs each product
li_containers =  page_soup.findAll("li",{"class":"gl-item"})
for i in range(0,len(li_containers)):
   p_name_div = li_containers[i].find("div",{"class":"p-name"})
   p_name = p_name_div.a.em.text.strip()
   print(p_name.strip(" "))
   f.write(p_name.strip(" "))
f.close()

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM