如何根据另一个 dataframe 中的列的值合并 dataframe 的两列？

Question

I have a dataframe called df_location:

location = {'location_id': [1,2,3,4,5,6,7,8,9,10],
            'temperature_value': [20,21,22,23,24,25,26,27,28,29],
            'humidity_value':[60,61,62,63,64,65,66,67,68,69]}
df_location = pd.DataFrame(locations)

I have another dataframe called df_islands:

islands = {'island_id':[10,20,30,40,50,60],
          'list_of_locations':[[1],[2,3],[4,5],[6,7,8],[9],[10]]}
df_islands = pd.DataFrame(islands)

Each island_id corresponds to one or more locations. As you can see, the locations are stored in a list. What I'm trying to do is to search the list_of_locations for each unique location and merge it to df_location in a way where each island_id will correspond to a specific location.

Final dataframe should be the following:

merged = {'location_id': [1,2,3,4,5,6,7,8,9,10],
                'temperature_value': [20,21,22,23,24,25,26,27,28,29],
                'humidity_value':[60,61,62,63,64,65,66,67,68,69],
                'island_id':[10,20,20,30,30,40,40,40,50,60]}
df_merged = pd.DataFrame(merged)

I don't know whether there is a method or function in python to do so. I would really appreciate it if someone can give me a solution to this problem.

Answer 1

The df.apply() method works here. It's a bit long-winded but it works:

df_location['island_id'] = df_location['location_id'].apply(
    lambda x: [
        df_islands['island_id'][i] \
        for i in df_islands.index \
        if x in df_islands['list_of_locations'][i]     
   
        # comment above line and use this instead if list is stored in a string
        # if x in eval(df_islands['list_of_locations'][i])        
        ][0]
)

First we select the final value we want if the if statement is True: df_islands['island_id'][i]

Then we loop over each column in df_islands by using df_islands.index

Then create the if statement which loops over all values in df_islands['list_of_locations'] and returns True if the value for df_location['location_id'] is in the list .

Finally, since we must contain this long statement in square brackets, it is a list. However, we know that there is only one value in the list so we can index it by using [0] at the end.

I hope this helps and happy for other editors to make the answer more legible!

print(df_location)

   location_id  temperature_value  humidity_value  island_id
0            1                 20              60         10
1            2                 21              61         20
2            3                 22              62         20
3            4                 23              63         30
4            5                 24              64         30
5            6                 25              65         40
6            7                 26              66         40
7            8                 27              67         40
8            9                 28              68         50
9           10                 29              69         60

Answer 2

The pandas method you're looking for to expand your df_islands dataframe is .explode(column_name) . From there, rename your column to location_id and then join the dataframes using pd.merge() . It'll perform a SQL-like join method using the location_id as the key.

import pandas as pd

locations = {'location_id': [1,2,3,4,5,6,7,8,9,10],
            'temperature_value': [20,21,22,23,24,25,26,27,28,29],
            'humidity_value':[60,61,62,63,64,65,66,67,68,69]}
df_locations = pd.DataFrame(locations)

islands = {'island_id':[10,20,30,40,50,60],
          'list_of_locations':[[1],[2,3],[4,5],[6,7,8],[9],[10]]}
df_islands = pd.DataFrame(islands)

df_islands = df_islands.explode(column='list_of_locations')

df_islands.columns = ['island_id', 'location_id']

pd.merge(df_locations, df_islands)

Out[]:
  location_id  temperature_value  humidity_value  island_id
0           1                 20              60         10
1           2                 21              61         20
2           3                 22              62         20
3           4                 23              63         30
4           5                 24              64         30
5           6                 25              65         40
6           7                 26              66         40
7           8                 27              67         40
8           9                 28              68         50
9          10                 29              69         60