[英]Adding data frame to excel sheet
I am trying to write a dataframe to excel using panda.ExcelWriter
after reading it from a huge csv file. 从巨大的csv文件读取数据panda.ExcelWriter
后,我尝试使用panda.ExcelWriter
将数据panda.ExcelWriter
写入excel。
This code updates the excel sheet but it doesn't appends the data to the excel which I want 这段代码更新了Excel工作表,但没有将数据追加到我想要的Excel中
import pandas as pd
reader = pd.read_csv("H:/ram/temp/1.csv", delimiter = '\t' ,chunksize = 10000, names = ['neo_user_id',
'gender',
'age_range',
'main_geolocation', # (user identifier of the client)
'interest_category_1',
'interest_category_2',
'interest_category_3',
'first_day_identifier'
], encoding="utf-8")
ew = pd.ExcelWriter('H:/ram/Formatted/SynthExport.xlsx', engine='xlsxwriter', options={'encoding':'utf-8'})
for chunks in reader:
chunks.to_excel(ew, 'Sheet1' , encoding = 'utf-8')
print len(chunks)
ew.save()
I also tried to use data.append()
and data.to_excel
doing this result is memory error. 我也尝试使用data.append()
和data.to_excel
来完成此结果是内存错误。 Since I am reading data in chunks is there any way to write the data to excel 由于我正在分块读取数据,因此有什么方法可以将数据写入excel
I got it working by this code 我通过此代码工作
import pandas as pd
import xlsxwriter
reader = pd.read_csv("H:/ram/user_action_export.2014.01.csv", delimiter = '\t', chunksize = 1000, names = ['day_identifier',
'user_id',
'site_id',
'device', # (user identifier of the client)
'geolocation',
'referrer',
'pageviews',
], encoding="utf-8")
startrows = 0
ew = pd.ExcelWriter('H:/ram/Formatted/ActionExport.xlsx', engine='xlsxwriter', options={'encoding':'utf-8'})
for chunks in reader:
chunks.to_excel(ew, 'Sheet1' , encoding = 'utf-8', startrow = startrows)
startrows = startrows + len(chunks)
print startrows
ew.save()
But still take so much time 但是还是要花很多时间
我不知道这是否是导致主要问题的原因,但是您不应该在块之间调用save()
,因为单次调用save()
关闭xlsxwriter
文件。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.