如何將多個csv文件中的行復制到pandas中的新文件？

Question

我有10個csv文件。 我想從所有csv文件中復制第一行並保存為新的csv文件，然后從所有csv文件中復制第二行並保存為第二個csv文件等。以下我的代碼僅針對第一行和其他rwos顯示NaN 。 我的錯誤在哪里？

碼

import pandas as pd
import datetime
import glob

path = r'/Jupyter_Works/new_csv'
all_files = glob.glob(path + "/*.csv")

date_time = datetime.datetime(2018, 1, 1)
index = pd.date_range(start='1/1/2018', periods= 8760, freq='H')

columns = ['Lat','Lon','Alt','Temperature','Relative Humidity','Wind speed','Wind direction','Short-wave irradiation']
dfcsv = pd.DataFrame(index=index, columns=columns)

for filename in all_files:
    df = pd.read_csv(filename, index_col='time', header=0)
    dfcsv.iloc[0] = df.iloc[0]

dfcsv

結果

Lat Lon Alt Temperature Relative Humidity   Wind speed  Wind direction  Short-wave irradiation
2018-01-01 00:00:00 31.03   49.36   99  285.56  52.82   2.95    128.5   0
2018-01-01 01:00:00 NaN NaN NaN NaN NaN NaN NaN NaN
2018-01-01 02:00:00 NaN NaN NaN NaN NaN NaN NaN NaN
2018-01-01 03:00:00 NaN NaN NaN NaN NaN NaN NaN NaN
2018-01-01 04:00:00 NaN NaN NaN NaN NaN NaN NaN NaN

Answer 1

首先創建一個帶有列表理解和concat大型DataFrame ，按照loc選擇的唯一值循環，並通過DataFrame.to_csv寫入文件。 它工作，因為每個DataFrame都有唯一索引，因此如果按唯一值選擇，則選擇所有文件中具有相同位置的行。

path = r'/home/nickan/Jupyter_Works/new_csv'
all_files = glob.glob(path + "/*.csv")

dfs = [pd.read_csv(fp, index_col='time', parse_dates=['time']) for fp in all_files]
df = pd.concat(dfs)

for x in df.index.unique():
    #removed duplicated index by index=False
    df.loc[x].to_csv(f'csv/file_{x.strftime("%Y-%m-%d_%H")}.csv', index=False)

編輯：

因為內存問題是可能的，所以使用dataFrames中每行循環的替代解決方案並以append模式寫入：

for i, fp in enumerate(all_files):
    df = pd.read_csv(fp, index_col='time', parse_dates=['time']) 
    for x in df.index:
        f = f'out/file_{x.strftime("%Y-%m-%d_%H")}.csv'
        if i == 0:
            df.loc[[x]].to_csv(f, index=False)

        else:
            df.loc[[x]].to_csv(f, index=False,header=None, mode='a')