Pandas divide two dataframe with different sizes

Question

I have a dataframe df1 as:

col1 col2 Val1 Val2
A    g    4    6
A    d    3    8
B    h    5    10
B    p    7    14

I have another dataframe df2 as:

col1 Val1 Val2
A    2    3
B    1    4

I want to divide df1 by df2 based on col1, val1 and val2 so that row A from df2 divides both rows A from df1 .

My final output of df1.div(df2) is as follows:

col1 col2 Val1 Val2
A    g    2    2
A    d    1.5  2
B    h    5    2.5
B    p    7    3.5

Answer 1

Convert col1 and col2 to MultiIndex , also convert col1 in second DataFrame to index and then use DataFrame.div :

df = df1.set_index(['col1', 'col2']).div(df2.set_index('col1')).reset_index()
#alternative with specify level of index
#df = df1.set_index(['col1', 'col2']).div(df2.set_index('col1'), level=0).reset_index()
print (df)
  col1 col2  Val1      Val2
0    A    g   2.0  2.000000
1    A    d   1.5  2.666667
2    B    h   5.0  2.500000
3    B    p   7.0  3.500000

Answer 2

I think there is a slight mistake in your example. For col Val2, 2nd row - 8/3 should be 2.67. So the final output df1.div(df2) should be :

  col1 col2  Val1      Val2
0    A    g   2.0  2.000000
1    A    d   1.5  2.666667
2    B    h   5.0  2.500000
3    B    p   7.0  3.500000

Anyways here is a possible solution:

1. Construct the 2 dfs

import pandas as pd

df1 = pd.DataFrame(data={'col1':['A','A','B','B'], 'col2': ['g','d','h','p'], 'Val1': [4,3,5,7], 'Val2': [6,8,10,14]}, columns=['col1','col2','Val1','Val2'])

df2 = pd.DataFrame(data={'col1':['A','B'], 'Val1': [2,1], 'Val2': [3,4]}, columns=['col1','Val1','Val2'])

print (df1)
print (df2)

Output:

>>>
col1 col2  Val1  Val2
0    A    g     4     6
1    A    d     3     8
2    B    h     5    10
3    B    p     7    14

  col1  Val1  Val2
0    A     2     3
1    B     1     4

Now we can just do an INNER JOIN of df1 and df2 on col: col1 . If you are not familiar with SQL joins have a look at this: sql-join . We can do join in pandas using the merge() method

## join df1, df2

merged_df = pd.merge(left=df1, right=df2, how='inner', on='col1')

print (merged_df)

Output:

>>>
col1 col2  Val1_x  Val2_x  Val1_y  Val2_y
0    A    g       4       6       2       3
1    A    d       3       8       2       3
2    B    h       5      10       1       4
3    B    p       7      14       1       4

Now that we have got the corresponding columns of df1 and df2 , we can simply compute the division and delete the redundant columns:

# Val1 = Val1_x/Val1_y, Val2 = Val2_x/Val2_y

merged_df['Val1'] = merged_df['Val1_x']/merged_df['Val1_y']
merged_df['Val2'] = merged_df['Val2_x']/merged_df['Val2_y']

# delete the cols: Val1_x,Val1_y,Val2_x,Val2_y

merged_df.drop(columns=['Val1_x', 'Val1_y', 'Val2_x', 'Val2_y'], inplace=True)

print (merged_df)

Final Output:

  col1 col2  Val1      Val2
0    A    g   2.0  2.000000
1    A    d   1.5  2.666667
2    B    h   5.0  2.500000
3    B    p   7.0  3.500000

I hope this solves your question :)

Answer 3

You can use the pandas.merge() function to execute a database-like join between dataframes , then use the result to divide column values:

# merge against col1 so we get a merged index
merged = pd.merge(df1[["col1"]], df2)
df1[["Val1", "Val2"]] = df1[["Val1", "Val2"]].div(merged[["Val1", "Val2"]])

This produces:

  col1 col2  Val1      Val2
0    A    g   2.0  2.000000
1    A    d   1.5  2.666667
2    B    h   5.0  2.500000
3    B    p   7.0  3.500000