Pandas multi-index unstack to single row

Question

I am pretty good with simple Pandas but am struggling with data reshaping and multi indices. I have a multindex dataframe that looks like so (it doesnt have to be a multindex but it seems the right thing to do)

name	index	f1	f2	f3	calc1	calc2	calc3
fox	1	red	white	fur	0.21	1.67	-0.34
	2				0.76	2.20	-1.02
	3				0.01	1.12	-0.22
chicken	1	white	yellow	feathers	0.04	1.18	-2.01
	2				0.18	0.73	-1.21
grain	1	yellow	bag	corn	0.89	1.65	-1.03
	2				0.34	2.45	-0.45
	3				0.87	1.11	-0.97

and all I want is:

name	f1	f2	f3	calc1_1	calc2_1	calc3_1	calc1_2	calc2_2	calc3_2	calc1_3	calc2_3	calc3_3
fox	red	white	fur	0.21	1.67	-0.34	0.76	2.20	-1.02	0.01	1.12	-0.22
chicken	white	yellow	feathers	0.04	1.18	-2.01	0.18	0.73	-1.21	NaN	NaN	NaN
grain	yellow	bag	corn	0.89	1.65	-1.03	0.34	2.45	-0.45	0.87	1.11	-0.97

I figure this has got to be an easy one for the pandas gurus out there. Thanks all for your help!!

Drew

Answer 1

Try set_index + unstack to reshape to long format

new_df = df.set_index(['name', 'index', 'f1', 'f2', 'f3']).unstack('index')

OR via pivot

new_df = df.pivot(index=['name', 'f1', 'f2', 'f3'], columns='index')

---

Sort MultiIndex with sort_index :

new_df = new_df.sort_index(axis=1, level=1)

---

Then reduce MultiIndex via map + reset_index :

new_df.columns = new_df.columns.map(lambda s: '_'.join(map(str, s)))

new_df = new_df.reset_index()

new_df :

      name      f1      f2        f3  calc1_1  calc2_1  calc3_1  calc1_2  calc2_2  calc3_2  calc1_3  calc2_3  calc3_3
0  chicken   white  yellow  feathers     0.04     1.18    -2.01     0.18     0.73    -1.21      NaN      NaN      NaN
1      fox     red   white       fur     0.21     1.67    -0.34     0.76     2.20    -1.02     0.01     1.12    -0.22
2    grain  yellow     bag      corn     0.89     1.65    -1.03     0.34     2.45    -0.45     0.87     1.11    -0.97

---

Complete Code:

import pandas as pd

df = pd.DataFrame({
    'name': ['fox', 'fox', 'fox', 'chicken', 'chicken', 'grain', 'grain',
             'grain'],
    'index': [1, 2, 3, 1, 2, 1, 2, 3],
    'f1': ['red', 'red', 'red', 'white', 'white', 'yellow', 'yellow', 'yellow'],
    'f2': ['white', 'white', 'white', 'yellow', 'yellow', 'bag', 'bag', 'bag'],
    'f3': ['fur', 'fur', 'fur', 'feathers', 'feathers', 'corn', 'corn', 'corn'],
    'calc1': [0.21, 0.76, 0.01, 0.04, 0.18, 0.89, 0.34, 0.87],
    'calc2': [1.67, 2.2, 1.12, 1.18, 0.73, 1.65, 2.45, 1.11],
    'calc3': [-0.34, -1.02, -0.22, -2.01, -1.21, -1.03, -0.45, -0.97]
})

new_df = (
    df.set_index(['name', 'index', 'f1', 'f2', 'f3'])
        .unstack('index')
        .sort_index(axis=1, level=1)
)

new_df.columns = new_df.columns.map(lambda s: '_'.join(map(str, s)))

new_df = new_df.reset_index()