Increment 3D array of counters from 2D list of indices (x,y,z) using: Python Numpy Slicing

Question

I want to increment a 3D matrix (nparray) of counters from a 2D array of events (x,y,t) The code below works:

TOF_cube=np.zeros((324,324,4095),np.int32) #initialise a 3d array for whole data set

data = np.fromfile(f, dtype='<i2', count=no_I16) #read all events, x,y,t as 1D array
data=data.reshape(events,cols)
xpos=data[:,0]
ypos=data[:,1]
tpos=data[:,2]
i=0
while i < events:              
    TOF_cube[xpos[i],ypos[i],tpos[i]] += 1
    i+=1

To use slicing and indexing I replace my while loop with

    TOF_cube[xpos,ypos,tpos] += 1

But rather than copying the correct 4365520 number of events (via the while loop and independently checked) I only record 4365197.

Why is the slicing method loosing events?

I am using exactly the same slices in the while loop and as an 'argument' to the index.

Answer 1

+= doesn't add twice if there are repeat indices.

To get an equivalent output in a vectorized manner, you'll need np.add.at :

np.add.at(TOF_cube, [xpos, ypos, tpos], 1)

Answer 2

As we do not know exactly what your data looks like it's hard to guess what the actual problem is. If this doesn't help please give an example that we can run ourselvs (ie without having the file f ).

Suppose you have x_pos = [1,1,2,3,5]

a = np.zeros(10)
for i in range(len(x_pos)):
    a[x_pos[i]]+=1
# gives a = array([ 0.,  2.,  1.,  1.,  0.,  1.,  0.,  0.,  0.,  0.])

However the other code

a[x_pos]+=1
# gives a = array([ 0.,  1.,  1.,  1.,  0.,  1.,  0.,  0.,  0.,  0.])

So if one of the indices occurs twice it is only updated once in the short version. Check whether this is actually the case in your xpos etc.

PS: I did a slightly easier version, with only one dimension, but the rules stay the same.