[英]Trying to export the data from the crawl to a csv file
I found this code online, and I want to use it, but I can't find a way to export the data collected to a csv file. 我在网上找到了此代码,并且想使用它,但是找不到将导出的数据导出到csv文件的方法。
import urllib
import scrapy
import json
import csv
from bs4 import BeautifulSoup
url = "http://www.straitstimes.com/tags/malaysia-crimes"
html = urllib.urlopen(url).read()
soup = BeautifulSoup(html)
# kill all script and style elements
for script in soup(["script", "style"]):
script.extract() # rip it out
# get text
text = soup.body.get_text()
# break into lines and remove leading and trailing space on each
lines = (line.strip() for line in text.splitlines())
# break multi-headlines into a line each
chunks = (phrase.strip() for line in lines for phrase in line.split(" "))
# drop blank lines
text = '\n'.join(chunk for chunk in chunks if chunk)
print(text)
The below appears to work for what i believe you want: 以下似乎适用于我认为您想要的:
I used the xlwt package to create, write to and save the workbook and then a loop to go through each line of text and write it to the workbook. 我使用xlwt包创建,写入和保存工作簿,然后使用循环遍历每一行文本并将其写入工作簿。 I saved it as testing.csv 我将其保存为testing.csv
import urllib
import scrapy
import json
import csv
from bs4 import BeautifulSoup
from xlwt import Workbook
url = "http://www.straitstimes.com/tags/malaysia-crimes"
html = urllib.urlopen(url).read()
soup = BeautifulSoup(html)
# create excel workbook
wb = Workbook()
sheet1 = wb.add_sheet('Sheet 1')
# kill all script and style elements
for script in soup(["script", "style"]):
script.extract() # rip it out
# get text
text = soup.body.get_text()
# break into lines and remove leading and trailing space on each
lines = (line.strip() for line in text.splitlines())
# break multi-headlines into a line each
chunks = (phrase.strip() for line in lines for phrase in line.split(" "))
# drop blank lines
text = '\n'.join(chunk for chunk in chunks if chunk)
print(text)
# go through each line and print to a new row in excel
counter = 1
for text_to_write in text.splitlines():
sheet1.write(counter,1,text_to_write)
counter = counter + 1
wb.save('testing.csv')
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.