將大熊貓數據框保存到excel

Question

我正在生成一個大數據框（以 CSV 格式保存時為 1.5 GB），並且需要將其存儲為 Excel 文件的工作表以及保存在單獨工作表中的第二個（小得多）數據框。

print('Reading temporaty files for variable {}:'.format(Var))
print(' Reading stations')
s=pd.read_csv(StatFile,sep=':',dtype={'ID': 'str'},encoding='utf-8')
print(' Reading data')
d=pd.read_csv(DataFile,sep=':',dtype='str',encoding='utf-8').transpose()
d.columns = d.iloc[0]
d=d[1:].astype('float')
d.reindex_axis(sorted(d.columns), axis=1)
print('Writing out Excel file for variable {}'.format(Var))
writer = pd.ExcelWriter(Path + Var + '.xlsx', engine='xlsxwriter')
d.to_excel(writer, sheet_name='Data')
OutStatCol=['ID','Name','Longitude','Latitude','GRS','OriginalVariable','VariableUnits','URL','JsonNode']
s.to_excel(writer, columns=OutStatCol, index=False, sheet_name='Stations')
writer.save()

我的代碼適用於較小的數據幀，但對於較大的數據幀，我收到以下錯誤：

Traceback (most recent call last):
  File "./Test2.py", line 29, in <module>
    writer.save()
  File "/home/user/miniconda2/lib/python2.7/site-packages/pandas/io/excel.py", line 1413, in save
    return self.book.close()
  File "/home/user/miniconda2/lib/python2.7/site-packages/xlsxwriter/workbook.py", line 297, in close
    self._store_workbook()
  File "/home/user/miniconda2/lib/python2.7/site-packages/xlsxwriter/workbook.py", line 624, in _store_workbook
    xlsx_file.write(os_filename, xml_filename)
  File "/home/user/miniconda2/lib/python2.7/zipfile.py", line 1148, in write
    self._writecheck(zinfo)
  File "/home/user/miniconda2/lib/python2.7/zipfile.py", line 1114, in _writecheck
    " would require ZIP64 extensions")
zipfile.LargeZipFile: Filesize would require ZIP64 extensions

有什么方法可以在 ExcelWriter 聲明或 to_excel() 方法中指定類似allowZip64=True的內容嗎？

謝謝！

Answer 1

這需要挖掘一些源代碼，但是......

print('Reading temporaty files for variable {}:'.format(Var))
print(' Reading stations')
s=pd.read_csv(StatFile,sep=':',dtype={'ID': 'str'},encoding='utf-8')
print(' Reading data')
d=pd.read_csv(DataFile,sep=':',dtype='str',encoding='utf-8').transpose()
d.columns = d.iloc[0]
d=d[1:].astype('float')
d.reindex_axis(sorted(d.columns), axis=1)
print('Writing out Excel file for variable {}'.format(Var))
writer = pd.ExcelWriter(Path + Var + '.xlsx', engine='xlsxwriter')

#THIS
writer.book.use_zip64()

d.to_excel(writer, sheet_name='Data')
OutStatCol=['ID','Name','Longitude','Latitude','GRS','OriginalVariable','VariableUnits','URL','JsonNode']
s.to_excel(writer, columns=OutStatCol, index=False, sheet_name='Stations')
writer.save()

應該工作

弄清楚writer沒有從工作簿中繼承，我花了比應有的時間更長的時間。 writer.book直接是一個工作簿實例......天啊

Answer 2

我剛剛在函數.to_excel()添加了engine='xlsxwriter'並解決了這個問題。