Apply function to every combination of columns

Question

I have this dataframe of values (ignore the numbers)

| A | B || C | D |
| 5 | 7 ||15 | 9 |
|13 | 12||15 | 9 |
|15 | 9 ||15 | 9 |
| 5 | 7 ||15 | 9 |
|13 | 12||15 | 9 |

I would like to apply a function(y,x) where (y,x) refers to every possible pair of columns in the dataframe (where y is not the same as x), then save the output from the function into a dictionary.

dict = {
  "A_B": ["blah", "blah", "blah"]
  "A_C": ["blah", "blah", "blah"]
  "A_D': ["blah", "blah", "blah"]
}

The dictionary shld store the output ["blah", "blah", "blah"] as a list in the dictionary.

Is there any way I can do this?

I suppose I have to make a for loop for every possible combination of the columns then apply the function to this, but I'm not sure how I can approach this. Appreciate any help. Thanks!

Answer 1

You can use itertools to get all the unique pairs of columns, and then create a loop to apply an operation to them:

import itertools

import pandas as pd
import numpy as np

df = pd.DataFrame(np.random.rand(4,4), columns=['A','B','C','D'])

output = {}
for x, y in itertools.combinations(df.columns, 2):
    output[f'{x}_{y}'] = (df[x] * df[y]).to_list()

print(output)

Result:

{'A_B': [0.5373750437559887,
  0.12077240904054208,
  0.02128148027667116,
  0.007578133428536173],
 'A_C': [0.039529837951733815,
  0.6965081691767399,
  0.04341804790548652,
  0.06767107788435821],
 'A_D': [0.07986784691848457,
  0.6510775100893785,
  0.05386515105603322,
  0.031171732070095028],
 'B_C': [0.03800661675931452,
  0.1710653815593833,
  0.136685425122361,
  0.07191805478874766],
 'B_D': [0.0767902629130131,
  0.15990741762557956,
  0.16957420765210682,
  0.033128042362621186],
 'C_D': [0.00564876743811126,
  0.9222041985664514,
  0.3459618868451074,
  0.2958261893932231]}

Here, the operation is multiplication * , but you could replace that with some other function, eg:

# find the maximum across the two columns
output = {}
for x, y in itertools.combinations(df.columns, 2):
    output[f'{x}_{y}'] = np.max(pd.concat([df[x], df[y]]))

print(output)
# {'A_B': 0.8884565587865458, 'A_C': 0.8687553149454967, 'A_D': 0.9452913147551872, 'B_C': 0.8884565587865458, 'B_D': 0.9452913147551872, 'C_D': 0.9452913147551872}