Columns to Rows in pandas

Question

I am trying to do the following operation in pandas . Any suggestions on pandas way of doing this?

In [1]: input  = pd.DataFrame({"X_1": [1], "X_2": [2], "X_3": [5], "Y_1": [1.2], "Y_2": [2.3], "Y_3": [3.4]})

In [2]: input
Out[2]: 
   X_1  X_2  X_3  Y_1  Y_2  Y_3
0    1    2    5  1.2  2.3  3.4

In [3]: output = pd.DataFrame({"X": [1,2,5], "Y": [1.2, 2.3, 3.4]})

In [4]: output
Out[4]: 
   X    Y
0  1  1.2
1  2  2.3
2  5  3.4

Answer 1

Use str.split and stack .

df.columns = df.columns.str.split('_', expand=True)
df.stack().loc[0]

   X    Y
1  1  1.2
2  2  2.3
3  5  3.4

Note: Index is [1, 2, 3] matching original columns.

Answer 2

Probably not the best answer, but you can do the following:

a = {"X_1": [1], "X_2": [2], "X_3": [5], "Y_1": [1.2], "Y_2": [2.3], "Y_3": [3.4]}
x = [a[key][0] for key in sorted(a.keys()) if 'X' in key]  
y = [a[key][0] for key in sorted(a.keys()) if 'Y' in key]
df = pd.DataFrame([x, y]).T

yielding:

  0 1 0 1.0 1.2 1 2.0 2.3 2 5.0 3.4 

Answer 3

You can first split columns by _ and create unique values a and b . Then create MultiIndex.from_product and stack :

cols = input.columns.str.split('_')
print (cols)
Index([['X', '1'], ['X', '2'], ['X', '3'], ['Y', '1'], 
       ['Y', '2'], ['Y', '3']], dtype='object')

a = cols.str[0].unique()
print (a)
['X' 'Y']

b = cols.str[1].unique()
print (b)
['1' '2' '3']

input.columns = pd.MultiIndex.from_product([a,b])
print (input.stack(1).reset_index(drop=True))
   X    Y
0  1  1.2
1  2  2.3
2  5  3.4

Answer 4

For this sort of thing, I prefer a melt followed by a string operation, followed by a pivot :

df = pd.melt(input)
df[['column', 'index']] = df['variable'].str.split('_', expand=True)
df = df.pivot(index='index', columns='column', values='value')
print(df)

output:

column    X    Y
index           
1       1.0  1.2
2       2.0  2.3
3       5.0  3.4