I have the following Pandas dataframe:
name1 name2
A B
A A
A C
A A
B B
B A
I want to add a column named new
which counts per groups of name1
how often name1
is the same as name2
.
Hence, the expected output is the following dataframe:
name1 name2 new
A B 2
A A 2
A C 2
A A 2
B B 1
B A 1
I have tried the following, but I get an error:
df['new'] = df.groupby('name1').apply(lambda x: (x[x['name1'] == x['name2']].fillna(False).sum()))
TypeError: incompatible index of inserted column with frame index
You can compare name1
with name2
, then group by name1
and sum
Trues :
df['new'] = df.name2.eq(df.name1).astype(int).groupby(df.name1).transform('sum')
df
# name1 name2 new
#0 A B 2
#1 A A 2
#2 A C 2
#3 A A 2
#4 B B 1
#5 B A 1
Or if using apply
, aggregate the counts firstly, then use map
to generate the new
column:
cnt = df.groupby('name1').apply(lambda g: (g.name1 == g.name2).sum())
df['new'] = df.name1.map(cnt)
Timing :
df = pd.concat([df]*10000)
%timeit df['new'] = df.name2.eq(df.name1).astype(int).groupby(df.name1).transform('sum')
# 100 loops, best of 3: 4.85 ms per loop
%%timeit
cnt = df.groupby('name1').apply(lambda g: (g.name1 == g.name2).sum())
df['new'] = df.name1.map(cnt)
# 10 loops, best of 3: 22.1 ms per loop
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.