根据python pandas数据帧中列的状态更改，将时间序列数据拆分为组

Question

I need to group some data in a pandas dataframe but the standard grouping method does not quite work how I need it to. It must group so that each change in "loc" and/or each change in "name" is treated as a separate group.

Example;

x = pd.DataFrame([['john','abc',1],['john','abc',2],['john','abc',3],['john','xyz',4],['john','xyz',5],['john','abc',6],['john','abc',7],['matt','abc',8]])
x.columns = ['name','loc','time']

name    loc  time
john    abc  1
john    abc  2
john    abc  3
john    xyz  4
john    xyz  5
john    abc  6
john    abc  7
matt    abc  8

I need to group these values so that the resulting data is

name    loc  first last
john    abc  1     3
john    xyz  4     5
john    abc  6     7
matt    abc  8     8

The default grouping function (correctly) groups all the loc and name values so we are only left with 3 groups (john / abc is 1 group). Does anybody know how the grouping can be forced to group how i require it to?

I'm able to generate the required table using a for loop (iterrows), but if there is a nice pandas pythonic way to do the same thing I would love to know.

Thank you in advance.

Matt

Answer 1

This is not really a job for groupby because the order of the rows matters. Instead, compare consecutive rows by using shift .

In [37]: cols = ['name', 'loc']

In [38]: change = (x[cols] != x[cols].shift(-1)).any(1).shift(1).fillna(True)

In [39]: groups = x[change]

In [40]: groups.columns = ['name', 'loc', 'first']

In [41]: groups['last'] = (groups['first'].shift(-1) - 1).fillna(len(x))

In [42]: groups
Out[42]:
   name  loc  first  last
0  john  abc      1     3
3  john  xyz      4     5
5  john  abc      6     7
7  matt  abc      8     8

[4 rows x 4 columns]

Answer 2

You can use a function in the groupby :

x = pd.DataFrame([['john','abc',1],['john','abc',2],['john','abc',3],['john','xyz',4],['john','xyz',5],['john','abc',6],['john','abc',7],['matt','abc',8]])
x.columns = ['name','loc','time']

last_group = None
c =0
def f(y):
    global c,last_group
    g = x.irow(y)['name'],x.irow(y)['loc']
    if last_group != g:
        c += 1
        last_group = g
    return c

print x.groupby(f).head()