我想在PANDAS数据框中计算每个主题内的观察次数

Question

I am quite new to using PANDAS and python in general.

I have a hierarchical data set with several subjects, each of whom have some number of observations. The total df is about half a million rows.

I want to calculate the observations number...

## toy problem

d = {'one' : Series(['a', 'a', 'a', 'b', 'b', 'b'], index = [0,1,2,3,4,5]),
     'two' : Series([1.1, 2.5, 3.3, 2.5, 3.3, 9.5], index = [0,1,2,3,4,5])}
df = DataFrame(d)

for i in df.one.unique():
    for j in range(0,len(df[df.one == i])):
        print j

So I want to assign j to a column for each row. I have no problem calculating j but I cannot figure out how to assign it. I have tried using iloc which is incredibly slow, or writing to a list and then joining this to the df, also really slow (currently running for over 30 mins and counting...). I understand that python is best with vectorised problems but I cannot think of a vector solution for this case.

What is the best way to do this? It is really easy and quick in R. I am currently migrating to Python & PANDAS under the expectation that it is faster but this doesnt appear to be the case here.

Any advice please?

Answer 1

You could use the GroupBy.cumcount method :

In [14]: df['j'] = df.groupby('one').cumcount()

In [15]: df
Out[15]: 
  one  two  j
0   a  1.1  0
1   a  2.5  1
2   a  3.3  2
3   b  2.5  0
4   b  3.3  1
5   b  9.5  2