Expand pandas dataframe and consolidate columns

Question

I have a dataframe that looks like this:

    desc    item    type1    date1    type2    date2    type3    date3
0   this    foo1      A        9/1      B        9/2      C        9/3
1   this    foo2      D        9/4      E        9/5      F        9/6

How do I get it to look like:

     desc    item    type    date
0    this    foo1      A      9/1
1    this    foo1      B      9/2
2    this    foo1      C      9/3
3    this    foo2      D      9/4
4    this    foo2      E      9/5
5    this    foo2      F      9/6

?

Answer 1

Check with wide_to_long

out = pd.wide_to_long(df.reset_index(), ['type','date'], i ='index', j = 'drop').reset_index(drop=True)
out
Out[127]: 
  type date
0    A  9/1
1    B  9/2
2    C  9/3

For your updated question, the same concept still applies, you just do not need to reset the index, since item is unique:

pd.wide_to_long(df, stubnames=['type','date'], i='item',j='drop').droplevel(-1).reset_index()



    item  type  date
0   foo1    A   9/1
1   foo2    D   9/4
2   foo1    B   9/2
3   foo2    E   9/5
4   foo1    C   9/3
5   foo2    F   9/6

Answer 2

You can also use .melt on two dataframes by passing a list to value_vars using list comprehension if a column contains type or date . Then, you can merge these two dataframes on the index:

df = pd.merge(df.melt(id_vars='item', value_vars=[col for col in df.columns if 'type' in col], value_name='type')[['item','type']],
              df.melt(id_vars='item', value_vars=[col for col in df.columns if 'date' in col], value_name='date')['date'],
              how='left', left_index=True, right_index=True).sort_values('type')
df
Out[1]: 
   item type date
0  foo1    A  9/1
2  foo1    B  9/2
4  foo1    C  9/3
1  foo2    D  9/4
3  foo2    E  9/5
5  foo2    F  9/6