[英]How do I merge multiple cvs files into one DataFrame in pandas?
I have a folder with a lot of csv-files each containing messurements of signal data.我有一个文件夹,里面有很多 csv 文件,每个文件都包含信号数据的混乱。 They have the following structure:
它们具有以下结构:
Frequency [kHz],Power [dbm]
852000,-135.812845793404
852008,-142.13849097071088
852016,-138.21218081816156
852024,-137.32593610384734
852032,-139.464539680863
I want to merge these files into a DataFrame with Frequency as the key column, because the frequency is the same in every file.我想将这些文件合并到一个 DataFrame 中,以频率为键列,因为每个文件中的频率都是相同的。 So it should look something like this in the DataFrame:
所以它在 DataFrame 中应该看起来像这样:
Frequency [kHz] | Power [dbm] | Power [dbm] | Power [dbm] | ...
So I wrote the following code:所以我写了以下代码:
df = pd.DataFrame()
for f in csv_files:
csv = pd.read_csv(f)
df = pd.merge(df, csv, on='Frequency [kHz]', sort=False)
But the only thing I get is an KeyError: 'Frequency [kHz]'
但我唯一得到的是
KeyError: 'Frequency [kHz]'
The closest I came to my desired result was through pd.concat([pd.read_csv(f) for f in csv_files], axis=0, sort=False)
but then there are still those Frequency columns in between.我最接近我想要的结果是通过
pd.concat([pd.read_csv(f) for f in csv_files], axis=0, sort=False)
但中间仍然有那些频率列。
You can read them all into a dictionary and use concat:您可以将它们全部读入字典并使用 concat:
import pandas as pd
import glob
path = 'path'
all_files = glob.glob(path + "/*.csv")
df_dict1 = {}
for filename in all_files:
df = pd.read_csv(filename)
df_dict1.update({f'{filename}':df})
df = pd.concat(df_dict1, axis =1)
df = df.droplevel(0, axis =1)
df.index = df['Frequency [kHz]']
df.drop(columns = 'Frequency [kHz]', inplace = True)
I think you can collect them all as dfs, and then merge, like so:我认为您可以将它们全部收集为 dfs,然后合并,如下所示:
data_frames = []
for f in csv_files:
df = pd.read_csv(f)
data_frames.append(df)
df_merged = reduce(lambda left, right: pd.merge(left, right, on=['Frequency [kHz]'],
how='outer'), data_frames)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.