I'm new to python and right now I'm trying to learn web scraping for personal travel planning. I would like to know after I print the result, how can I output it into a table format or CSV format.
right now the result goes like the following:
{'price': '115', 'name': 'The hotel name'}
{'price': '97', 'name': 'the hotel name'}
.......
I Googled the method of some modules like pandas and prettytable, find it is too much challenge for me to understand. So I am here to see if there is any solutions regarding my problem.
The code is as below:
import requests
from bs4 import BeautifulSoup
url="http://hotelname.com/arrivalDate=05%2F23%2F2016**&departureDate=05%2F24%2F2016" #means arrive on May23 and leaves on May
wb_data = requests.get(url)
soup = BeautifulSoup(wb_data.text,'lxml')
names = soup.select('.PropertyName')
prices = soup.select('.RateSection ')
for name,price in zip(names,prices):
data = {
"name":name.get_text(),
"price":price.get_text()
}
print (data)`
I hope your previous codes are right and the writing part;
import csv
with open('filename.csv', 'w') as csvfile: #create csv file with w mode
fieldnames = ['names', 'prices'] #header names
writer = csv.DictWriter(csvfile, fieldnames=fiel dnames) #set the writer
writer.writeheader()
for name,price in zip(names,prices): #writing the elements
writer.writerow({'names': name.get_text(), 'prices': price.get_text()})
You could use pandas to create a DataFrame and then write the DataFrame to csv. This would require a slight change to your dict to make it easier, the code could look something like this:
import pandas as pd
#...
data = {
0: {"name": name.get_text()},
1: {"price": price.get_text()}
}
df = pd.DataFrame.from_dict(data, orient='index')
df.to_csv('filename.csv', index=False)
You just need to write to the file, where you are printing the values:
import requests
import csv
from bs4 import BeautifulSoup
url="http://hotelname.com/arrivalDate=05%2F23%2F2016**&departureDate=05%2F24%2F2016" #means arrive on May23 and leaves on May
wb_data = requests.get(url)
soup = BeautifulSoup(wb_data.text,'lxml')
names = soup.select('.PropertyName')
prices = soup.select('.RateSection ')
with open('results.csv', 'w') as outfile:
writer = csv.writer(outfile, delimiter=',')
writer.writerow(['Name', 'Price']]
for name,price in zip(names,prices):
writer.writerow([name.get_text(), price.get_text()])
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.