numpy diff weird behaviour

Question

I have a case of strange behaviour of numpy diff:

a = list(img_arr[y_coord_1,:])
print a
print np.diff(a)

>>[62, 62, 62, 62, 62, 62, 62, 62, 63, 62, 96, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 98, 66, 63, 64, 64, 63, 63, 63, 63, 63, 63, 63, 63, 63, 63, 64, 65, 65, 64, 63, 63, 63, 63, 63, 63, 63, 64, 64, 63, 63, 63, 63, 63, 64, 65, 65, 64, 63, 63, 63, 63]
>>[  0   0   0   0   0   0   0   1 255  34   2   0   0   0   0   0   0  0
0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0
0   0   0   0   0   0   0   0   0   0   0   0   0 224 253   1   0 255
0   0   0   0   0   0   0   0   0   1   1   0 255 255   0   0   0   0
0   0   1   0 255   0   0   0   0   1   1   0 255 255   0   0   0]

Now, when I run this in the terminal I get the correct answer of

array([  0,   0,   0,   0,   0,   0,   0,   1,  -1,  34,   2,   0,   0,
     0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
     0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,   0,
     0,   0,   0,   0,   0,   0,   0,   0,   0,   0, -32,  -3,   1,
     0,  -1,   0,   0,   0,   0,   0,   0,   0,   0,   0,   1,   1,
     0,  -1,  -1,   0,   0,   0,   0,   0,   0,   1,   0,  -1,   0,
     0,   0,   0,   1,   1,   0,  -1,  -1,   0,   0,   0])

What sort of things can cause this - I am using a few other compiled libraries in this script, if this of relevance

Edit: I've just spotted that its the negative numbers that are wrong - and the upper limit is very suspicious. Looks like a dtype issue..

Answer 1

.tolist() is a better way of converting an array to a list (or nested lists). It carries the conversion all the way down. list() just iterates on one level. And since an array is already iterable, I don't think list(anarray) does anything useful.

start with an array:

In [789]: z
Out[789]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8)
In [790]: type(list(z)[0])
Out[790]: numpy.uint8

list() is the same as this list comprehension:

In [791]: type([i for i in z][0])
Out[791]: numpy.uint8

the correct list conversion

In [792]: type(z.tolist()[0])
Out[792]: int

Why were you using list() in the first place? You didn't need it for diff . If overflow is an issue, dtype conversion is better.

np.diff will turn a list back into an array before taking the differences.

In [793]: np.diff(z.tolist())
Out[793]: array([1, 1, 1, 1, 1, 1, 1, 1, 1])
In [794]: np.diff(list(z))
Out[794]: array([1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=uint8)
In [795]: np.diff(z.astype('int'))
Out[795]: array([1, 1, 1, 1, 1, 1, 1, 1, 1])
In [796]: np.diff(z)
Out[796]: array([1, 1, 1, 1, 1, 1, 1, 1, 1], dtype=uint8)
In [797]: np.array(list(z))
Out[797]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=uint8)