如何在按 MultiIndex 名称选择时分配给 Pandas DataFrame？

Question

Main question : How do I select/slice a multi-indexed DataFrame, using the name of the MultiIndex level, in a way that allows me to assign to that slice?

Test Data

data = io.StringIO('''Fruit,Color,Count,Price
Apple,Red,3,$1.29
Apple,Green,9,$0.99
Pear,Red,25,$2.59
Pear,Green,26,$2.79
Lime,Green,9999,$0.39
''')
df_fruit = pd.read_csv(data, index_col=['Fruit', 'Color'])

new_green_data = io.StringIO('''Fruit,Count,Price
Apple,2,$0.96
Lime,9993,$0.40
Pear,12,$2.90
''')
df_new_green = pd.read_csv(new_green_data, index_col='Fruit')

This sets up two DataFrames:

df_fruit :

| Fruit   | Color   |   Count | Price   |
|:--------|:--------|--------:|:--------|
| Apple   | Red     |       3 | $1.29   |
| Apple   | Green   |       9 | $0.99   |
| Pear    | Red     |      25 | $2.59   |
| Pear    | Green   |      26 | $2.79   |
| Lime    | Green   |    9999 | $0.39   |

df_new_green :

| Fruit   |   Count | Price   |
|:--------|--------:|:--------|
| Apple   |       2 | $0.96   |
| Lime    |    9993 | $0.40   |
| Pear    |      12 | $2.90   |

The Want

I want to update the rows in df_fruit , in which Color is Green , so that they match the values in the incoming df_new_green data. The final output should be:

| Fruit   | Color   |   Count | Price   |
|:--------|:--------|--------:|:--------|
| Apple   | Red     |       3 | $1.29   |
| Apple   | Green   |       2 | $0.96   |
| Pear    | Red     |      25 | $2.59   |
| Pear    | Green   |      12 | $2.90   |
| Lime    | Green   |    9993 | $0.40   |

Note that the order of the fruits in df_new_green differs from df_fruit . Thus, when performing assignment, I need to preserve the indices of both sides so that it's handled correctly.

What I Know

I know several ways to select what I want to update in the DataFrame:

df_fruit.xs(key='Green', level='Color')

This produces the right view of the data, but I can't assign to it. Similarly close:

df_fruit[df_fruit.index.get_level_values('Color') == 'Green']

and

idx = pd.IndexSlice
df_fruit.loc[idx[:, 'Green'], :]

both give me the same view, but they still include the Color level of the MultiIndex:

| Fruit   | Color   |   Count | Price   |
|:--------|:--------|--------:|:--------|
| Apple   | Green   |       9 | $0.99   |
| Pear    | Green   |      26 | $2.79   |
| Lime    | Green   |    9999 | $0.39   |

I can assign to this view using df_new_green , but this yields NaN s because the df_new_green does not include the Color level in its index. The second choice (using IndexSlice ) is also not great because I'm not selecting the level based on its name, but rather its position in the MultiIndex. If I run droplevel('Green') on either one, again I get the right view but I can't assign to it.

I could drop the index on the new values, but this leads do the wrong values being used:

df_fruit.loc[idx[:, 'Green'], :] = df_new_green._values

This yields:

| Fruit   | Color   |   Count | Price   |
|:--------|:--------|--------:|:--------|
| Apple   | Red     |       3 | $1.29   |
| Apple   | Green   |       2 | $0.96   |
| Pear    | Red     |      25 | $2.59   |
| Pear    | Green   |    9993 | $0.40   |
| Lime    | Green   |      12 | $2.90   |

...but this is wrong because the Pear and Lime values got swapped. I need to preserve the index(es) on the update DataFrame.

The Ugly Way

df_fruit[df_fruit.index.get_level_values('Color') == 'Green'] = df_new_green.assign(Color='Green').set_index('Color', append=True)

...guh. This yields the right answer and meets the requirements, but holy cow that's ugly.

Answer 1

I'd use assign and set_index then combine_first :

(df_new_green.assign(Color='Green')
             .set_index('Color', append=True)
             .combine_first(df_fruit))

Output:

|    | Fruit   | Color   |   Count | Price   |
|---:|:--------|:--------|--------:|:--------|
|  0 | Apple   | Green   |       2 | $0.96   |
|  1 | Apple   | Red     |       3 | $1.29   |
|  2 | Lime    | Green   |    9993 | $0.40   |
|  3 | Pear    | Green   |      12 | $2.90   |
|  4 | Pear    | Red     |      25 | $2.59   |

Answer 2

The solution is to:

• Add Green as the second level of the index in df_new_green , setting its name to Color .
• Update df_fruit (in-place) with this (temporary) DataFrame.

The code to do it is:

df_fruit.update(df_new_green.set_index(pd.Index(
    ['Green'] * df_new_green.index.size, name='Color'), append=True))

Answer 3

Not very nice but it does what it takes.

new_prices = []
for index, row in df_fruit.iterrows():
    if index[1] == 'Green':
        price = df_new_green.loc[index[0], ['Price']].values[0]
        new_prices.append(price)
    else:
        new_prices.append(row['Price'])

df_fruit['Price'] = new_prices

Output:

             Count  Price
Fruit Color              
Apple Red        3  $1.29
      Green      9  $0.96
Pear  Red       25  $2.59
      Green     26  $2.90
Lime  Green   9999  $0.40