numpy 阵列中的滤波元件具有一定的最小频率

Question

I have an array of 5000 counts and need to do a chisquare test on it. However the test only works when the expected frequency is >5 for every value. I have found the frequency of each value in the dataset using collections.Counter(x) and I can see that there are some values with a frequency of 1 or 2. Now I would like to remove any value with a frequency<5 from my original dataset x but I don't know how to do this.

Once I have removed these points, I need to create an expected poisson distribution to use in the chisquare test, but once again making sure that the expected frequency is >5. I've made some distributions using stats.poisson.rvs but is there a way I can make sure that the frequency is always above 5? Or would it be best to create the distribution, and go through the steps in the first part of my question?

Answer 1

One way to filter your array to certain frequency (eg >5 ) is ( a is your original array):

#this method assumes array a is consists of integers
a[np.in1d(a, np.where(np.bincount(a)>5)[0])]

Another way is:

#works for non-integer arrays
values, counts = np.unique(a, return_counts=True)
a[np.in1d(a, values[counts>5])]

I would guess bincount solution is faster.