Python numpy.where and structured (record) array

Question

I'd like to use numpy.where with my array of records :

new_array = np.core.records.fromrecords([(data[0],data[1],data[2],data[3],data[4],data[5],ND‌​ate)],names='Date, Name, Age, Start, End,Avg,NDate',formats='S10,d,d,d,d,d,d')

I've tried this:

np.where(work.stock==6.26)
>>(array([], dtype=int32),)

Returned nothing, so I tried this:

np.where(work.stock==work.stock[6.26])
>>(array([6]),)

Returned number of the equal elements. But I just need the index of the first occurance. How can it be done?

Answer 1

From your first statement I can construct a look alike array

new_array = np.core.records.fromrecords([(1,2,3,4,5,6,7)],
   names='Date, Name, Age, Start, End,Avg,NDate',
   formats='S10,d,d,d,d,d,d')

which displays as

rec.array([('1', 2.0, 3.0, 4.0, 5.0, 6.0, 7.0)], 
    dtype=[('Date', 'S10'), ('Name', '<f8'), ('Age', '<f8'), 
    ('Start', '<f8'), ('End', '<f8'), ('Avg', '<f8'), ('NDate', '<f8')])

but your

np.where(work.stock==6.26)

leaves me scratching my head. What is work ? Is stock a field of another array. It's not a field of new_array .

np.where(work.stock==work.stock[6.26])

is equally puzzling. How can you index an array with a float (6.26)?

Be careful about doing tests like work.stock==6.26 . Equality tests on floats often do not work, due to rounding errors.

Practice with np.where on simple arrays. Start with integer ones before trying floats. Learn what it produces when there aren't any matches, and when there are multiple ones.

If there is only one record in your new_array it might be better if you used simple Python lists.

---

Here's how you could use where to find the 1st record with a given value in a specified field

Define a small array with 2 fields, 4 records, and integer values (for ease of testing):

In [482]: X=np.core.records.fromrecords([(1,2),(2,3),(3,3),(4,1),(1,2)],dtype='i,i')

In [483]: X
Out[483]: 
rec.array([(1, 2), (2, 3), (3, 3), (4, 1), (1, 2)], 
      dtype=[('f0', '<i4'), ('f1', '<i4')])

In [484]: X['f0'] 
Out[484]: array([1, 2, 3, 4, 1])

The elements of X['f0'] that are equal to 1 (integer)

In [486]: i=np.where(X['f0']==1)

In [487]: i
Out[487]: (array([0, 4]),)

And the index of the first one (the first [0] picks the array out of the tuple)

In [488]: i[0][0]
Out[488]: 0

In [489]: X[i[0][0]]
Out[489]: (1, 2)

Alternatively I could use the whole i to get all the X that meet this criteria

In [494]: X[i]
Out[494]: 
rec.array([(1, 2), (1, 2)], 
      dtype=[('f0', '<i4'), ('f1', '<i4')])

and select the first:

In [495]: X[i][0]
Out[495]: (1, 2)

If there aren't any matches, then where returns an empty array

In [496]: i=np.where(X['f0']==5)

In [497]: X[i]
Out[497]: 
rec.array([], 
      dtype=[('f0', '<i4'), ('f1', '<i4')])

A where test on a structured array field is really no different than a test on a row or column of a 2d array.

---

I changed X.f0 to X['f0'] because the record array notation can be confusing, and non-record structured arrays are more common these days, especially when arrays are generated by np.getfromtxt .

X==2 returns False . X['f0']==2 returns array([False, True, False, False, False], dtype=bool) . Boolean tests do not work across fields of a structured array.

To find 'rows' of X across all fields, I have to test against a matching structured array, eg

In [507]: X==np.array([(2,3)],dtype=X.dtype)
Out[507]: rec.array([False,  True, False, False, False], dtype=bool)

np.where can work with that Out[507] boolean array.

---

Find X records where one field or the other has a 2 :

In [518]: I=(X['f0']==2) | (X['f1']==2)

In [519]: I
Out[519]: array([ True,  True, False, False,  True], dtype=bool)

In [520]: X[I]
Out[520]: 
rec.array([(1, 2), (2, 3), (1, 2)], 
      dtype=[('f0', '<i4'), ('f1', '<i4')])

I have to test each field individually.

In this case, all fields of X are integer, so I can construct a 2d integer view, and perform a test on that:

In [526]: np.any(X.view(int).reshape(X.shape[0],-1)==2,axis=1)
Out[526]: array([ True,  True, False, False,  True], dtype=bool)

See recent https://stackoverflow.com/a/33094425/901925 for more on using a view of a structured array.