how to count occurrence of each unique value in pandas

Question

I have large pandas dataframe, I would like to count the occurrence of each unique value in it, I try following but it takes to much time and memory usage. How can I do it in a pythonic way?

pack=[]
for index,row in packets.iterrows ():
    pack.extend(pd.Series(row).dropna().values.tolist())

unique, count= np.unique(pack, return_counts=True)
counts= np.asarray((unique, count))

Answer 1

It seems like you want to compute value counts across all columns . You can flatten it to a series, drop NaNs, and call value_counts . Here's a sample -

df

     a    b
0  1.0  NaN
1  1.0  NaN
2  3.0  3.0
3  NaN  4.0
4  5.0  NaN
5  NaN  4.0
6  NaN  5.0

pd.Series(df.values.ravel()).dropna().value_counts()

5.0    2
4.0    2
3.0    2
1.0    2
dtype: int64

Another method is with np.unique -

u, c = np.unique(pd.Series(df.values.ravel()).dropna().values, return_counts=True)
pd.Series(c, index=u)

1.0    2
3.0    2
4.0    2
5.0    2
dtype: int64

Note that the first method sorts results in descending order of counts, while the latter does not.