将熊猫数据框与带日期的多索引合并

Question

I have some dataframes with a date index from multiple sources which I want to merge into a single multiindex dataframe. I'm struggling to figure out how to do this.

Starting with two dataframes:

Source 1

+---------------------+------+------+-----+-------+
|        date         | open | high | low | close |
+---------------------+------+------+-----+-------+
| 2018-04-04 20:00:00 | xxx  | xxx  | xxx | xxx   |
| 2018-04-04 21:00:00 | xxx  | xxx  | xxx | xxx   |
| 2018-04-04 22:00:00 | xxx  | xxx  | xxx | xxx   |
+---------------------+------+------+-----+-------+

Source 2

+---------------------+------+------+-----+-------+
|        date         | open | high | low | close |
+---------------------+------+------+-----+-------+
| 2018-04-04 20:00:00 | xxx  | xxx  | xxx | xxx   |
| 2018-04-04 21:00:00 | xxx  | xxx  | xxx | xxx   |
| 2018-04-04 22:00:00 | xxx  | xxx  | xxx | xxx   |
+---------------------+------+------+-----+-------+

I'd like to merge them so they are multiindexed on the date with the source1 or source2.

Something like:

+---------------------+---------+------+-----+-------+
|                     |         |      |     |       |
+---------------------+---------+------+-----+-------+
| 2018-04-04 20:00:00 | source1 |      |     |       |
|                     | open    | high | low | close |
|                     | xxx     | xxx  | xxx | xxx   |
|                     | source2 |      |     |       |
|                     | open    | high | low | close |
|                     | xxx     | xxx  | xxx | xxx   |
| 2018-04-04 21:00:00 | source1 |      |     |       |
|                     | open    | high | low | close |
|                     | xxx     | xxx  | xxx | xxx   |
|                     | source2 |      |     |       |
|                     | open    | high | low | close |
|                     | xxx     | xxx  | xxx | xxx   |
| 2018-04-04 22:00:00 | source1 |      |     |       |
|                     | open    | high | low | close |
|                     | xxx     | xxx  | xxx | xxx   |
|                     | source2 |      |     |       |
|                     | open    | high | low | close |
|                     | xxx     | xxx  | xxx | xxx   |
+---------------------+---------+------+-----+-------+

Can anyone help?

Thanks!

Answer 1

You can go for concat specifying the keys ie

df3 = pd.concat([df1,df2],keys=['source1','source2']).reset_index(level=0)

df3 = df3.set_index(['date','level_0']).sort_index(level='date')



                                open    high    low    close
 date                 level_0                                
 2018-04-04 20:00:00  source1   xxx     xxx     xxx    xxx   
                      source2   xxx     xxx     xxx    xxx   
 2018-04-04 21:00:00  source1   xxx     xxx     xxx    xxx   
                      source2   xxx     xxx     xxx    xxx   
 2018-04-04 22:00:00  source1   xxx     xxx     xxx    xxx   
                      source2   xxx     xxx     xxx    xxx   

Answer 2

Use concat with keys and set_index for DatetimeIndex and then swaplevel with sort_index :

df = (pd.concat([df1.set_index('date'),df2.set_index('date')], keys=['source1','source2'])
        .swaplevel(0,1)
        .sort_index())
print (df)
                            open high  low close
date                                            
2018-04-04 20:00:00 source1  xxx  xxx  xxx   xxx
                    source2  xxx  xxx  xxx   xxx
2018-04-04 21:00:00 source1  xxx  xxx  xxx   xxx
                    source2  xxx  xxx  xxx   xxx
2018-04-04 22:00:00 source1  xxx  xxx  xxx   xxx
                    source2  xxx  xxx  xxx   xxx