多个pandas数据帧的交集

Question

I have a number of dataframes (100) in a list as:

frameList = [df1,df2,..,df100]

Each dataframe has the two columns DateTime , Temperature .

I want to intersect all the dataframes on the common DateTime column and get all their Temperature columns combined/merged into one big dataframe: Temperature from df1, Temperature from df2, Temperature from df3, .., Temperature from df100.

(pandas merge doesn't work as I'd have to compute multiple (99) pairwise intersections).

Answer 1

Use pd.concat , which works on a list of DataFrames or Series.

pd.concat(frameList, axis=1, join='inner')

This is better than using pd.merge , as pd.merge will copy the data pairwise every time it is executed. pd.concat copies only once. However, pd.concat only merges based on an axes, whereas pd.merge can also merge on (multiple) columns.

Answer 2

you can try using reduce functionality in python..something like this

dfs = [df0, df1, df2, dfN]
df_final = reduce(lambda left,right: pd.merge(left,right,on='DateTime'), dfs)

Answer 3

You could iterate over your list like this:

df_merge = frameList[0]
for df in frameList[1:]:       
    df_merge = pd.merge(df_merge, df, on='DateTime', how='inner')