熊猫：基于列数据合并或联接数据框？

Question

I am trying to add several columns of data to an existing dataframe. The dataframe itself was built from a number of other dataframes, which I successfully joined on indices, which were identical. For that, I used code like this:

    data = p_data.join(r_data)

I actually joined these on a multi-index, so the dataframe looks something like the following, where Name1 and Name 2 are indices:

    Name1    Name2    present    r      behavior
    a        1        1          0      0
             2        1          .5     2
             4        3          .125   1
    b        2        1          0      0
             4        5          .25    4
             8        1          0      1

So the Name1 index does not repeat data, but the Name2 index does (I'm using this to keep track of dyads, so that Name1 & Name2 together are only represented once). What I now want to add are 4 columns of data that correspond to Name2 data (information on the second member of the dyad). Unlike the "present" "r" and "behavior" data, these data are per individual, not per dyad. So I don't need to consider Name1 data when merging.

The problem is that while Name2 data are repeated to exhaust the dyad combos, the "Name2" column in the data I would now like to add only has one piece of data per Name2 individual:

    Name2    Data1    Data2    Data3
    1        80       6        1
    2        61       8        3
    4        45       7        2
    8        30       3        6

What I would like the output to look like:

    Name1    Name2    present    r      behavior    Data1    Data2    Data3
    a        1        1          0      0           80       6        1
             2        1          .5     2           61       8        3
             4        3          .125   1           45       7        2
    b        2        1          0      0           61       8        3
             4        5          .25    4           45       7        2
             8        1          0      1           30       3        6

Despite reading the documentation, I am not clear on whether I can use join() or merge() for the desired outcome. If I try a join to the existing dataframe like the simple one I've used previously, I end up with the new columns but they are full of NaN values. I've also tried various combinations using Name1 and Name2 as either columns or as indices, with either join or merge (not as random as it sounds, but I'm clearly not interpreting the documentation correctly!). Your help would be very much appreciated, as I am presently very much lost.

Answer 1

I'm not sure if this is the best way, but you could use reset_index to temporarily make your original DataFrame indexed by Name2 only. Then you could perform the join as usual. Then use set_index to again make Name1 part of the MultiIndex:

import pandas as pd

df = pd.DataFrame({'Name1':['a','a','a','b','b','b'],
                   'Name2':[1,2,4,2,4,8],
                   'present':[1,1,3,1,5,1]})
df.set_index(['Name1','Name2'], inplace=True)

df2 = pd.DataFrame({'Data1':[80,61,45,30],
                    'Data2':[6,8,7,3]},
                   index=pd.Series([1,2,4,8], name='Name2'))
result = df.reset_index(level=0).join(df2).set_index('Name1', append=True)
print(result)
#              present  Data1  Data2
# Name2 Name1                       
# 1     a            1     80      6
# 2     a            1     61      8
#       b            1     61      8
# 4     a            3     45      7
#       b            5     45      7
# 8     b            1     30      3

To make the result look even more like your desired DataFrame, you could reorder and sort the index:

print(result.reorder_levels([1,0],axis=0).sort(axis=0))
#              present  Data1  Data2
# Name1 Name2                       
# a     1            1     80      6
#       2            1     61      8
#       4            3     45      7
# b     2            1     61      8
#       4            5     45      7
#       8            1     30      3