Flag changing column in a groupby DataFrame using Pandas

Question

Here is the dataset:

>>> df = pd.DataFrame({'id_police':['p123','p123','p123','b123','b123'],
                   'date':['24/01/2017','24/11/2017','25/02/2018','24/02/2018','24/03/2018'],
                   'prime':[0,0,10,20,30],
                   'prime2':[0,30,10,20,0],
})
###
  id_police        date  prime  prime2
0      p123  24/01/2017      0       0
1      p123  24/11/2017      0      30
2      p123  25/02/2018     10      10
3      b123  24/02/2018     20      20
4      b123  24/03/2018     30       0

This is the result I got when I used the working solution from @Erfan:

  id_police        date  prime  prime2  changed
0      p123  24/01/2017      0       0<-      0
1      p123  24/11/2017      0<-    30<-      1
2      p123  25/02/2018     10<-    10        1
3      b123  24/02/2018     20      20        0
4      b123  24/03/2018     30       0        0

The command:

df['changed'] = (df[['prime', 'prime2']].shift().eq(0).any(axis=1) & df[['prime', 'prime2']].ne(0).any(axis=1)).astype(int)

Now I want to apply this for each id_police like add a groupby or something... Thanks for help!

Answer 1

We can acces the groupid and groups in our groupby object and then make the changed columns in each iteration:

groups = []
for _, grp in df.groupby('id_police'):
    grp['changes'] = (grp[['prime', 'prime2']].shift().eq(0).any(axis=1) & grp[['prime', 'prime2']].ne(0).any(axis=1)).astype(int)
    groups.append(grp)

df_final = pd.concat(groups).sort_index()

---

Which yields

print(df_final)
  id_police        date  prime  prime2  changes
0      p123  24/01/2017      0       0        0
1      p123  24/11/2017      0      30        1
2      p123  25/02/2018     10      10        1
3      b123  24/02/2018     20      20        0
4      b123  24/03/2018     30       0        0

---

If you want to turn off the SetCopyWarning , use the following:

pd.options.mode.chained_assignment = None

Answer 2

Based On your command:

cols = ['prime', 'prime2']

df['changed'] = (df.groupby('id_police', sort=False, as_index=False)
                   .apply(lambda x: (x[cols].ne(0) & x[cols].shift(1).eq(0))
                   .any(axis=1).astype(int))
                   .reset_index(drop=True))
df

  id_police        date  prime  prime2  changed
0      p123  24/01/2017      0       0        0
1      p123  24/11/2017      0      30        1
2      p123  25/02/2018     10      10        1
3      b123  24/02/2018     20      20        0
4      b123  24/03/2018     30       0        0

Use groupby with apply to apply function on each group. and set sort=False to make it order the same with your main df.