将DataFrame附加到多索引DataFrame

Question

I have a DataFrame with three indices that looks like this:

                                               stat1             stat2
sample                        env  run                                                  
sample1                       0    0          36.214             71
                                   1          31.808             71
                                   2          28.376             71
                                   3          20.585             71
sample2                       0    0           2.059             29
                                   1           2.070             29
                                   2           2.038             29

This represents a process that runs on different samples of data. This process is run multiples times in different environments, which qualifies the results.

It may sound simple, but I am having trouble trying to add a new environment result as a DataFrame :

            stat1          stat2
run                                                  
0           0.686             29
1           0.660             29
2           0.663             29

This should be indexed under df.loc[["sample1", 1]] . I have tried this:

df.loc[["sample1", 1]] = result

And using DataFrame.append . But the first just raises a KeyError and the second does not seem to modify the DataFrame at all.

What am I missing here?

Edit: adding that when using append like df.loc["sample"].append(result) the problem is that it messes up the multi-index. It is transformed into a single index where the former multi-index is merged into a tuples, like (0, 0) or (0, 1) standing for environment 0, run 1, and so on; and the index of the appended DataFrame (a ranged index representing each run) becomes the new unwanted index.

Answer 1

The core of the issue here is the difference in the indexes. One way to overcome this would be to change result's index to include the 0,1 levels to be set, then use concat to append the datataframe. see the example below

In [68]: result.index = list(zip(["sample1"]*len(result), [1]*len(result),result
    ...: .index))

In [69]: df = pd.concat([df,result])
         df
Out[69]: 
                  stat1  stat2
sample  env run               
sample1 0   0    36.214     71
            1    31.808     71
            2    28.376     71
            3    20.585     71
sample2 0   0     2.059     29
            1     2.070     29
            2     2.038     29
sample1 1   0     0.686     29
            1     0.660     29
            2     0.663     29

Edit: Once the index is change, you can even use append

In [21]: result.index = list(zip(["sample1"]*len(result), [1]*len(result),result
    ...: .index))

In [22]: df.append(result)
Out[22]: 
                  stat1  stat2
sample  env run               
sample1 0   0    36.214     71
            1    31.808     71
            2    28.376     71
            3    20.585     71
sample2 0   0     2.059     29
            1     2.070     29
            2     2.038     29
sample1 1   0     0.686     29
            1     0.660     29
            2     0.663     29