仅使用pandas在组内获取重复值

Question

I have a data frame such as :

groups  ids numbers
group3  id4 89
group1  id1 50
group1  id1 30
group1  id2 90
group2  id4 89
group2  id6 76
group3  id4 90

and the idea it to find with groupby groups the duplicated ids and get a new data frame with only duplicated ids by groups such as:

group1  id1 50
group1  id1 30
group3  id4 89
group3  id4 90

I tried:

for groups in df.groupby('groups'):
 print(df['ids'].duplicated)

Thanks for your help.

Answer 1

Function groupby is not necessary, for better performance use DataFrame.duplicated by multiple columns and parameter keep=False for get all dupes, then filter by boolean indexing :

df = df[df.duplicated(['groups','ids'], keep=False)]
print (df)
   groups  ids  numbers
0  group3  id4       89
1  group1  id1       50
2  group1  id1       30
6  group3  id4       90

If sorting necessary add DataFrame.sort_values with DataFrame.reset_index for default index:

df = (df[df.duplicated(['groups','ids'], keep=False)]
         .sort_values(['groups','ids'])
         .reset_index(drop=True))
print (df)
   groups  ids  numbers
0  group1  id1       50
1  group1  id1       30
2  group3  id4       89
3  group3  id4       90

Answer 2

You can use:

df.groupby('groups').apply(lambda x: \
            x[x.duplicated('ids',keep=False)]).reset_index(drop=True)

Output:

   groups  ids  numbers
0  group1  id1       50
1  group1  id1       30
2  group3  id4       89
3  group3  id4       90