pandas – apply function to DataFrame row-wise, resulting in a new DataFrame of different dimensions

Question

I want to be able to apply a function to a DataFrame (row-wise) such that it can return a new DataFrame that does not necessarily have the same dimensions or indexing as the original.

Let's say I have a DataFrame , df :

    A   B   C   D
0  A0  B0  C0  D0
1  A1  B1  C1  D1
2  A2  B2  C2  D2
3  A3  B3  C3  D3

and a function foo() :

>>> def foo(series):
...  series['E'] = 'NEW_STUFF'
...  series['F'] = 'MORE_NEW_STUFF'
...  df = pd.DataFrame(series.drop('B')).transpose()
...  return pd.concat([df,df], keys='qw')
... 

such that

>>> foo(df.iloc[0])
      A   C   D          E               F
q 0  A0  C0  D0  NEW_STUFF  MORE_NEW_STUFF
w 0  A0  C0  D0  NEW_STUFF  MORE_NEW_STUFF

I want to apply foo() to df such that it results in a new DataFrame where the results of running foo() on each row are stacked into a single DataFrame , kind of like

      A   C   D          E               F
q 0  A0  C0  D0  NEW_STUFF  MORE_NEW_STUFF
w 0  A0  C0  D0  NEW_STUFF  MORE_NEW_STUFF
q 1  A1  C1  D1  NEW_STUFF  MORE_NEW_STUFF
w 1  A1  C1  D1  NEW_STUFF  MORE_NEW_STUFF
q 2  A2  C2  D2  NEW_STUFF  MORE_NEW_STUFF
w 2  A2  C2  D2  NEW_STUFF  MORE_NEW_STUFF
q 3  A3  C3  D3  NEW_STUFF  MORE_NEW_STUFF
w 3  A3  C3  D3  NEW_STUFF  MORE_NEW_STUFF

However, running df.apply(foo, axis=1) does not return this. Instead I get

>>> df.apply(foo, axis=1)
0          A   C   D          E               F
q 0...
1          A   C   D          E               F
q 1...
2          A   C   D          E               F
q 2...
3          A   C   D          E               F
q 3...
dtype: object

What do I need to modify above to get the results I'm looking for?

Answer 1

Try with

pd.concat([foo(y) for _,y in df.iterrows()])
Out[64]: 
      A   C   D          E               F
q 0  A0  C0  D0  NEW_STUFF  MORE_NEW_STUFF
w 0  A0  C0  D0  NEW_STUFF  MORE_NEW_STUFF
q 1  A1  C1  D1  NEW_STUFF  MORE_NEW_STUFF
w 1  A1  C1  D1  NEW_STUFF  MORE_NEW_STUFF
q 2  A2  C2  D2  NEW_STUFF  MORE_NEW_STUFF
w 2  A2  C2  D2  NEW_STUFF  MORE_NEW_STUFF
q 3  A3  C3  D3  NEW_STUFF  MORE_NEW_STUFF
w 3  A3  C3  D3  NEW_STUFF  MORE_NEW_STUFF