Iteratively build multi-index dataframe in pandas

Question

I have n small dataframes which I combine into one multiindex dataframe.

d1 = DataFrame(np.array([
    ['a', 5, 9],
    ['b', 4, 61],
    ['c', 24, 9]]),
    columns=['name', 'attr11', 'attr22'])

d2 = DataFrame(np.array([
    ['a', 5, 19],
    ['b', 14, 16],
    ['c', 4, 9]]),
    columns=['name', 'attr21', 'attr22'])

d3 = DataFrame(np.array([
    ['a', 5, 19],
    ['b', 14, 16],
    ['d', 10, 14],
    ['c', 4, 9]]),
    columns=['name', 'attr21', 'attr22'])

How to combine these into one dataframe?

Result:

      name attr11 attr21  attr22
d1    a    5    NULL    9
      b    4    NULL    61
      c    24   NULL    9
d2    a    NULL   5    19
      b    NULL   14   16
      c    NULL   4    9
d3    a    NULL   5    19 
      b    NULL   14   16
      c    NULL   4    9
      d    NULL   10   14

Answer 1

you can build the multiindex after the concatenation. You just need to add a column to each frame with the dataframe id:

frames =[d1,d2,d3]

Add a column to each frame with the frame id:

for x in enumerate(frames):
    x[1]['frame_id'] = 'd'+str(x[0]+1)

then concatenate the list of frames and set the index on the desired columns:

pd.concat(frames).set_index(['frame_id','name'])