将数据帧第一行的单元格值与其他行的单元格值进行比较

Question

I have a datafarme which has 50 columns and above 200 rows with binary values:

a1  a2  a3  a4  ….. a50
0   1   0   1   ….. 1
1   0   0   1   ….  0
0   1   1   0   ….  0
1   1   1   0   ….  1

I would like to compare cell values of first row to other rows one by one and make the 51th column which output the non-matching cells as below: (since the first row is not compared with any row it will get a nan value)

a51
NAN
a1,a2,…,a50
a3,a4…,a50
a1,a3,a4,…

I am not sure how to do this efficiently. I have not find any answer similar to this question. Sorry if I am asking repeated question. Thank you in advance!

Answer 1

Setup

import numpy as np
df = pd.DataFrame(np.random.randint(2,size=(200,50)),
                  columns =[f'a{i}' for i in range(1,51)])

---

Series.dot + DataFrame.add_suffix and Series.str.rstrip

df['a51']=df.iloc[1:].ne(df.iloc[0]).dot(df.add_suffix(', ').columns).str.rstrip(', ')

Time comparision for 50 columns and 200 rows

%%timeit
df['a51'] = df.iloc[1:].ne(df.iloc[0]).dot(df.add_suffix(', ').columns).str.rstrip(', ')
25.4 ms ± 681 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)


%%timeit
a = df.to_numpy()
m = np.where(a[0,:] != a[1:,None], df.columns, np.nan)
pd.DataFrame(m.squeeze()).stack().groupby(level=0).agg(', '.join)
41.1 ms ± 4.16 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)


%%timeit
df.iloc[1:].apply(lambda row: df.columns[df.iloc[0] != row].values, axis=1)
147 ms ± 18.9 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

Answer 2

Here's one approach:

import numpy as np

a = df.to_numpy()
m = np.where(a[0,:] != a[1:,None], df.columns, np.nan)
pd.DataFrame(m.squeeze()).stack().groupby(level=0).agg(', '.join)

0    a1, a2, a50
1    a3, a4, a50
2     a1, a3, a4
dtype: object

---

Input data:

print(df)

   a1  a2  a3  a4  a50
0   0   1   0   1    1
1   1   0   0   1    0
2   0   1   1   0    0
3   1   1   1   0    1

Answer 3

I assume you want the list of column names that don't match the first row:

df['a51'] = df.iloc[1:].apply(lambda row: df.columns[df.iloc[0] != row].values, axis=1)

200 rows is small enough so that apply(..., axis=1) is not a performance concern.