How to rearrange table in pandas in a format suitable for analysis in R?

Question

In pandas:

df = pd.DataFrame({'row1':['a','b','a','a','b','b','a','b','b','a'], 'row2':['x','x','y','y','y','x','x','y','x','y'],'col':[1,2,1,2,2,1,2,1,1,2],'val':[34,25,22,53,33,19,42,38,33,61]})

p = pd.pivot_table(df,values='val',rows=['row1','row2'],cols='col')

col         1   2
row1 row2        
a    x     34  42
     y     22  57
b    x     26  25
     y     38  33

Is it possible to reshape the pivot table in a data frame suitable for analysis in R?, something of the type:

  row1 row2 col val
    a   x   1   34
    a   y   1   22
    a   x   2   42
    a   y   2   57      
    b   x   1   26  
    b   y   1   38
    b   x   2   25
    b   y   2   33

Answer 1

One way is to use stack :

In [11]: p.stack()
Out[11]: 
row1  row2  col
a     x     1      34
            2      42
      y     1      22
            2      57
b     x     1      26
            2      25
      y     1      38
            2      33
dtype: int64

This doesn't give a name attribute, so you have to set it as you reset the index :

In [12]: df = p.stack().reset_index(name='val')

In [13]: df
Out[13]: 
  row1 row2  col  val
0    a    x    1   34
1    a    x    2   42
2    a    y    1   22
3    a    y    2   57
4    b    x    1   26
5    b    x    2   25
6    b    y    1   38
7    b    y    2   33

Answer 2

You can do:

>>> pd.melt(p.reset_index(), id_vars=['row1', 'row2'],
            var_name='col', value_name='val')
  row1 row2  col  val
0    a    x    1   34
1    a    y    1   22
2    b    x    1   26
3    b    y    1   38
4    a    x    2   42
5    a    y    2   57
6    b    x    2   25
7    b    y    2   33

Which resets the index, making row1 and row2 columns:

>>> p.reset_index()
col row1 row2   1   2
0      a    x  34  42
1      a    y  22  57
2      b    x  26  25
3      b    y  38  33

And then pd.melt melts in all columns except row1 and row2 ( id_vars parameter) into one column named val ( value_name parameter) and keeps the information to which column they belonged in the column named col ( var_name parameter).