[英]Saving scraped result set into a CSV file
我編寫了一個小腳本,它采用 ebay 結果集並將每個字段存儲在不同的變量中:鏈接、價格、出價。
如何獲取變量並將每個拍賣項目的每個結果保存到一個 CSV 文件中,其中每一行代表一個不同的拍賣項目?
例如:鏈接、價格、出價
到目前為止,這是我的代碼:
import requests, bs4
import csv
import requests
import pandas as pd
res = requests.get('http://www.ebay.com/sch/i.html?LH_Complete=1&LH_Sold=1&_from=R40&_sacat=0&_nkw=gerald%20ford%20autograph&rt=nc&LH_Auction=1&_trksid=p2045573.m1684')
res.raise_for_status()
soup=bs4.BeautifulSoup(res.text)
# grabs the link, selling price, and # of bids from historical auctions
links = soup.find_all(class_="vip")
prices = soup.find_all("span", "bold bidsold")
bids = soup.find_all("li", "lvformat")
這應該可以完成這項工作:
import csv
import requests
import bs4
res = requests.get('http://www.ebay.com/sch/i.html?LH_Complete=1&LH_Sold=1&_from=R40&_sacat=0&_nkw=gerald%20ford%20autograph&rt=nc&LH_Auction=1&_trksid=p2045573.m1684')
res.raise_for_status()
soup = bs4.BeautifulSoup(res.text)
# grab all the links and store its href destinations in a list
links = [e['href'] for e in soup.find_all(class_="vip")]
# grab all the bid spans and split its contents in order to get the number only
bids = [e.span.contents[0].split(' ')[0] for e in soup.find_all("li", "lvformat")]
# grab all the prices and store those in a list
prices = [e.contents[0] for e in soup.find_all("span", "bold bidsold")]
# zip each entry out of the lists we generated before in order to combine the entries
# belonging to each other and write the zipped elements to a list
l = [e for e in zip(links, prices, bids)]
# write each entry of the rowlist `l` to the csv output file
with open('ebay.csv', 'w') as csvfile:
w = csv.writer(csvfile)
for e in l:
w.writerow(e)
結果,您將獲得一個 .csv 文件,該文件以,
(逗號)作為分隔符。
import requests, bs4
import numpy as np
import requests
import pandas as pd
res = requests.get('http://www.ebay.com/sch/i.html? LH_Complete=1&LH_Sold=1&_from=R40&_sacat=0&_nkw=gerald%20ford%20autograph&r t=nc&LH_Auction=1&_trksid=p2045573.m1684')
res.raise_for_status()
soup=bs4.BeautifulSoup(res.text, "lxml")
# grabs the link, selling price, and # of bids from historical auctions
df = pd.DataFrame()
l = []
p = []
b = []
for links in soup.find_all(class_="vip"):
l.append(links)
for bids in soup.find_all("li", "lvformat"):
b.append(bids)
for prices in soup.find_all("span", "bold bidsold"):
p.append(prices)
x = np.array((l,b,p))
z = x.transpose()
df = pd.DataFrame(z)
df.to_csv('/Users/toasteez/ebay.csv')
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.