Suppose I have a dataset that looks something like:
INDEX A B C
1 1 1 0.75
2 1 1 1
3 1 0 0.35
4 0 0 1
5 1 1 0
I want to get a dataframe that looks like the following, with the original columns, and all possible interactions between columns:
INDEX A B C A_B A_C B_C
1 1 1 0.75 1 0.75 0.75
2 1 1 1 1 1 1
3 1 0 0.35 0 0.35 0
4 0 0 1 0 0 0
5 1 1 0 1 0 0
My actual datasets are pretty large (~100 columns). What is the fastest way to achieve this?
I could, of course, do a nested loop, or similar, to achieve this but I was hoping there is a more efficient way.
You could use itertools.combinations for this:
>>> import pandas as pd
>>> from itertools import combinations
>>> df = pd.DataFrame({
... "A": [1,1,1,0,1],
... "B": [1,1,0,0,1],
... "C": [.75,1,.35,1,0]
... })
>>> df.head()
A B C
0 1 1 0.75
1 1 1 1.00
2 1 0 0.35
3 0 0 1.00
4 1 1 0.00
>>> for col1, col2 in combinations(df.columns, 2):
... df[f"{col1}_{col2}"] = df[col1] * df[col2]
...
>>> df.head()
A B C A_B A_C B_C
0 1 1 0.75 1 0.75 0.75
1 1 1 1.00 1 1.00 1.00
2 1 0 0.35 0 0.35 0.00
3 0 0 1.00 0 0.00 0.00
4 1 1 0.00 1 0.00 0.00
If you need to vectorize an arbitrary function on the pairs of columns you could use:
import numpy as np
def fx(x, y):
return np.multiply(x, y)
for col1, col2 in combinations(df.columns, 2):
df[f"{col1}_{col2}"] = np.vectorize(fx)(df[col1], df[col2])
I am not aware of a native pandas
function to solve this, but itertools.combinations
would be an improvement over a nested loop.
You could do something like:
from itertools import combinations
df = pd.DataFrame(data={"A": [1,1,1,0,1],
"B": [1,1,0,0,1],
"C": [0.75, 1, 0.35, 1, 0]})
for comb in combinations(df.columns, 2):
col_name = comb[0] + "_" + comb[1]
result[col_name] = df[comb[0]] * df[comb[1]]
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.