Pandas: reindex multi-level columns after insertion

Question

Starting from a two-level column dataframe I add a column at the second level the following way:

for metric in ('test_EER', 'test_AUC'):
    baseline = summary_df[metric]['lower_confidence_bound']['dummy']
    summary_df[metric, 'improvement'] = (summary_df[metric]['lower_confidence_bound'] -baseline)/baseline

which gives me a unordered column indexing looking like:

How can I reindex the columns so that the ('test_AUC','improvement') fits before ('test_EER'). Also I do not want to change the ordering mean, std, lcb, improvement.

Answer 1

For you case, manual arrangement would work:

cols = pd.MultiIndex.from_product((['test_AUC', 'test_EER'],
                                   ['mean', 'std', 'lower_bound', 'improvement']
                                  )
                                 )

df[cols]

Output:

  test_AUC                             test_EER                            
      mean std lower_bound improvement     mean std lower_bound improvement
0        0   0           0         0.6        0   0           0         0.1

Answer 2

Generalizing on @QuangHoang's answer:

cols = pd.MultiIndex.from_product( df.columns.levels)
df = df[cols]

would do the sorting for other column labels.