Python Pandas concatenate dataframes and rename index

Question

I have three dataframes, all of which have 50 columns and one row. The same column names are used in each dataframe, and the single row is always indexed as 0. I'm trying to concatenate them to make viewing and comparing the data easier.

features = pd.concat([raw_features, fea_features, transformed_features], axis=0)

Now I want to rename the rows. I've tried several things including:

features = pd.concat([raw_features, fea_features, transformed_features], axis=0).reindex(['Raw_pulltest', 'FEA', 'Transformed_pulltest'])

which gives the error cannot reindex from a duplicate axis

and

features = pd.concat([raw_features, fea_features, transformed_features], axis=0).reset_index().reindex(['Raw_pulltest', 'FEA', 'Transformed_pulltest'])

which gives me the structure I want, except all values are now nan .

Please can you help me rename the index on the concatenated dataframe?

Answer 1

Use keys parameter in pd.concat :

Try this:

pd.concat([raw_features, fea_features, transformed_features], 
          axis=0, keys=['Raw_pulltest', 'FEA', 'Transformed_pulltest'])\
  .reset_index(level=1, drop=True)

Example:

d1 = pd.DataFrame([[1,1,1]],index=[0])

d2 = pd.DataFrame([[2,2,2]],index=[0])

d3 = pd.DataFrame([[3,3,3]], index=[0])

pd.concat([d1,d2,d3],axis=0, keys=['d1','d2','d3']).reset_index(level=1, drop=True)

Output:

    0  1  2
d1  1  1  1
d2  2  2  2
d3  3  3  3