Python：识别列中的连续零，删除它们的行并开始新编号

Question

I have the following df and I want to split the data into trips.

In: df = pd.DataFrame([[1001,0.054012973,0],[1001,0.44923679,12],[1001,0,1],[1001,0,1],[1001,0.44676617,2],
[1001,1.8310822,1],[1001,0,1],[1001,0,11],[1001,0,1],[1001,0,20],[1001,0,1],[1001,0,54],[1001,10.0604029,2],
[1001,11.642113,0],[1001,0,1],[1002,0,2],[1002,1.23463449,23],[1002,1.8310822,1],[1002,0,1]],
columns=['Dev_ID','Speed','Duration'])

out:    Dev_ID  Speed   Duration
    0   1001    0.054013    0
    1   1001    0.449237    12
    2   1001    0.000000    1
    3   1001    0.000000    1
    4   1001    0.446766    2
    5   1001    1.831082    1
    6   1001    0.000000    1
    7   1001    0.000000    11
    8   1001    0.000000    1
    9   1001    0.000000    20
    10  1001    0.000000    1
    11  1001    0.000000    54
    12  1001    10.060403   2
    13  1001    11.642113   0
    14  1001    0.000000    1
    15  1002    0.000000    2
    16  1002    1.234634    23
    17  1002    1.831082    1
    18  1002    0.000000    1

The criteria for the splitting is having a speed value of 0 longer than 120 sec. So I have to go for each dev_ID and somehow check if there are consecutive zeros that last more than 120 sec. If the condition is true, I want to delete these rows (where zeros last more than 120 sec) and start a new id in the trip_ID column. So the results should look like this:

    Dev_ID  Speed   Duration    Trip_ID
0   1001    0.054013    0   10
1   1001    0.449237    12  10
2   1001    0.000000    1   10
3   1001    0.000000    1   10
4   1001    0.446766    2   10
5   1001    1.831082    1   10
6   1001    10.060403   2   11
7   1001    11.642113   0   11
8   1001    0.000000    1   11
9   1002    0.000000    2   12
10  1002    1.234634    23  12
11  1002    1.831082    1   12
12  1002    0.000000    1   12

Answer 1

I not totally sure I understood the condition, but I made a generic code that hopefully will be similar and you can adapt.

The key ideas are: using pd.shift() to get the difference, use np.where to get a list of indexes where sequences of speed difference = 0, split those indexes in contigous groups with get_contigous_index, then for every contigous group if the sum of duration is > 120 then change 'Trip_id'

I assumed your duration is in minutes, otherwise none of the intervals would be greater than 120

import pandas as pd
import numpy as np
from itertools import groupby
from operator import itemgetter
df = pd.DataFrame([[1001,0.054012973,0],[1001,0.44923679,12],[1001,0,1],[1001,0,1],[1001,0.44676617,2],
[1001,1.8310822,1],[1001,0,1],[1001,0,11],[1001,0,1],[1001,0,20],[1001,0,1],[1001,0,54],[1001,10.0604029,2],
[1001,11.642113,0],[1001,0,1],[1002,0,2],[1002,1.23463449,23],[1002,1.8310822,1],[1002,0,1]],
columns=['Dev_ID','Speed','Duration'])
df['Duration'] = df['Duration']*60
df['Trip_ID'] = df['Dev_ID']

def get_contigous_index(indexes):
    ranges = []
    for k,g in groupby(enumerate(indexes),lambda x:x[0]-x[1]):
        group = (map(itemgetter(1),g))
        group = list(map(int,group))
        ranges.append((group[0],group[-1]))
    return ranges

for Dev_ID, data in df.groupby("Dev_ID"):
    data['speed_diff'] = data['Speed'] - data['Speed'].shift(1)
    diff_0 = np.where(data['speed_diff'] == 0)[0]

    for contigousZeroes_range in get_contigous_index(diff_0):
        fst_idx, lst_idx = list(contigousZeroes_range)
        range_ = list(range(fst_idx,lst_idx+1))
        subgroup = data.loc[range_ ,data.columns]
        if not subgroup.empty:
            if subgroup['Duration'].sum() > 120:
                df.loc[range_,'Trip_ID'] = "a_different_id"
print(df)

this will print an dataframe like this:

    Dev_ID      Speed  Duration         Trip_ID
0     1001   0.054013         0            1001
1     1001   0.449237       720            1001
2     1001   0.000000        60            1001
3     1001   0.000000        60            1001
4     1001   0.446766       120            1001
5     1001   1.831082        60            1001
6     1001   0.000000        60            1001
7     1001   0.000000       660  a_different_id
8     1001   0.000000        60  a_different_id
9     1001   0.000000      1200  a_different_id
10    1001   0.000000        60  a_different_id
11    1001   0.000000      3240  a_different_id
12    1001  10.060403       120            1001
13    1001  11.642113         0            1001
14    1001   0.000000        60            1001
15    1002   0.000000       120            1002
16    1002   1.234634      1380            1002
17    1002   1.831082        60            1002
18    1002   0.000000        60            1002

Answer 2

Based on the suggested solution from @Dataman (many thanks) the code that worked for me is:

for Dev_ID, data in df.groupby("Dev_ID"): 
    for k, g in groupby(data.iterrows(), lambda x: x[1]['Speed']): #group consecutive speeds
        l = list(g)
        if l[0][1]['Speed'] == 0: # check if the consective speeds are zeros
           dur = sum(x[1]['Duration'] for x in l) # calculate how long speed 0 lasts
           if dur>120:
              zeros_idx.append([x[0] for x in l]) # save indexes where speed = 0 for long time
df.drop((item for sublist in zeros_idx for item in sublist),axis=0,inplace=True) #delete long stops