从 numpy 数组中删除小于 1 的元素

Question

I am trying to plot large data (50 million values) but I am getting a MemoryError . Now I am trying to clear my dataset from redundant values. In my case, these are all values below 1 and above -1 . One thing to keep in mind is that the plot should look the same as the original, but without the noise. Is there a better way to do this than using loops or list comprehensions ?

Original Plot:

Edit:

Thank you for the replies. If I use the proposed approach:

daty = daty[(-1 > daty) | (daty > 1)]

It results in this:

Answer 1

如果您的数组名为data ：

clipped_data = data[(-1 > data) | (data > 1)]

Answer 2

Numpy can do addressing based on an array of booleans, so you can do stuff like this, if you want to set the unwanted values to 0:

badIndices = (myArray > 1) | (myArray < -1)
myArray[badIndices]=0

Answer 3

Numpy allows you to apply a mask to any single dimension of an array, resulting in a subset of the unmasked rows (or data points in your case).

data = np.array([1.5, 0.2, -5, -0.5])
mask = np.abs(data) > 1
data = data[mask]
print(data) # output: [1.5, -5]

Note that -5 now is at position 1 after the filter.

Answer 4

A sample I used on imbedded array

import numpy as np

data = np.array([[0, 3], [2, 15], [15, 7], [3, 6]])
e1 = (data[:, 0] < 15)
e2 = (data[:, 1] < 15)

print(data[e1 & e2])
# array([[0, 3], [3, 6]])