为什么“ numpy.ndarray.view”会忽略先前对“ numpy.ndarray.newbyteorder”的调用？

Question

I have a NumPy array with one element of data type uint32 :

>>> import numpy as np
>>> a = np.array([123456789], dtype=np.uint32)
>>> a.dtype.byteorder
'='

Then, I can choose to interpret the data as little-endian:

>>> a.newbyteorder("<").dtype.byteorder
'<'
>>> a.newbyteorder("<")
array([123456789], dtype=uint32)

Or as big-endian:

>>> a.newbyteorder(">").dtype.byteorder
'>'
>>> a.newbyteorder(">")
array([365779719], dtype=uint32)

Where the latter returns a different number 365779719 as my platform is little-endian - and therefore has been written to the memory in little-endian order.

Now, what's unexpected for me is the fact that a following appended call to view seems to be unaffected by this interpretation:

>>> a.newbyteorder("<").view(np.uint8)
array([ 21, 205,  91,   7], dtype=uint8)
>>> a.newbyteorder(">").view(np.uint8)
array([ 21, 205,  91,   7], dtype=uint8)

I would have expected the numbers to be the other way round for the big-endian byte order. Why doesn't this happen? Doesn't view view the data "through" the newbyteorder method?

By the way: if I use byteswap instead of newbyteorder and therefore copy and change the bytes in the memory, I obviously get the desired result:

>>> a.byteswap("<").view(np.uint8)
array([ 21, 205,  91,   7], dtype=uint8)
>>> a.byteswap(">").view(np.uint8)
array([  7,  91, 205,  21], dtype=uint8)

However, I don't want to copy the data.

Answer 1

The new byte order applied with newbyteorder is solely a property of the array's dtype; a.newbyteorder("<") returns a view of a with a little-endian dtype. It doesn't change the contents of memory, and it doesn't affect the array's shape or strides.

ndarray.view doesn't care about the original array's dtype, little-endian or big. It cares about the array's shape, strides, and actual memory content, none of which have changed.

Answer 2

Just to add to @user2357112's answer , from documentation :

As you can imagine from the introduction, there are two ways you can affect the relationship between the byte ordering of the array and the underlying memory it is looking at:

• Change the byte-ordering information in the array dtype so that it interprets the underlying data as being in a different byte order . This is the role of arr.newbyteorder()
• Change the byte-ordering of the underlying data , leaving the dtype interpretation as it was. This is what arr.byteswap() does.

My emphasis in the quote above.

---

Other thought gathered from comments:

Since newbyteorder() is similar to view() in that it just changes the interpretation of the underlying data without changing the data, it appears that a view into a view is a view to the same (original) data. So, yes, you cannot "chain" views (well, you can... but it is always a view to the same original data).

How do I get the uint8 chunks in big-endian order without changing the memory, then?

Try np.sum(a.newbyteorder('<')) (alternatively, try a.newbyteorder('<').tolist() ) and also change sign/endianness. So, my answer to the above question would be that you can't do that: either the memory is changed "in-place" with byteswap() or by making a copy of data to a new memory location when accessing the elements in the view.

Answer 3

In [280]: a = np.array([123456789, 234567891, 345678912], dtype=np.uint32)

In [282]: a.tobytes()
Out[282]: b'\x15\xcd[\x07\xd38\xfb\r@\xa4\x9a\x14'

In [284]: a.view('uint8')
Out[284]: 
array([ 21, 205,  91,   7, 211,  56, 251,  13,  64, 164, 154,  20],
      dtype=uint8)

This is the same as a.view('<u1') and a.view('>u1') since endedness doesn't matter with single bytes.

In [291]: a.view('<u4')
Out[291]: array([123456789, 234567891, 345678912], dtype=uint32)
In [292]: a.view('>u4')
Out[292]: array([ 365779719, 3543726861, 1084529172], dtype=uint32)

A view depends entirely on the data, not on the current (last) view:

In [293]: a.view('<u4').view('u1')
Out[293]: 
array([ 21, 205,  91,   7, 211,  56, 251,  13,  64, 164, 154,  20],
      dtype=uint8)
In [294]: a.view('>u4').view('u1')
Out[294]: 
array([ 21, 205,  91,   7, 211,  56, 251,  13,  64, 164, 154,  20],
      dtype=uint8)

About the idea of reshaping and reversing:

In [295]: a.view('u1').reshape(-1,4)
Out[295]: 
array([[ 21, 205,  91,   7],
       [211,  56, 251,  13],
       [ 64, 164, 154,  20]], dtype=uint8)
In [296]: a.view('u1').reshape(-1,4)[:,::-1]
Out[296]: 
array([[  7,  91, 205,  21],
       [ 13, 251,  56, 211],
       [ 20, 154, 164,  64]], dtype=uint8)

But I can't change the view (to u4 ) of this array because it isn't contiguous:

In [297]: a.view('u1').reshape(-1,4)[:,::-1].view('<u4')
....
ValueError: To change to a dtype of a different size, the array must be C-contiguous

Look a bit more at the properties of this reversed array:

In [298]: a1 = a.view('u1').reshape(-1,4)[:,::-1]
In [299]: a1.flags
Out[299]: 
  C_CONTIGUOUS : False
  F_CONTIGUOUS : False
  ....
In [300]: a1.strides             # reversing is done with strides
Out[300]: (4, -1)

The 2 arrays share the same databuffer. a2 just starts at a different byte:

In [301]: a.__array_interface__['data']
Out[301]: (32659520, False)
In [302]: a1.__array_interface__['data']
Out[302]: (32659523, False)

I can't do an inplace shape change of a1 :

In [304]: a1.shape = (12,)
...
AttributeError: incompatible shape for a non-contiguous array

If I do a reshape , I get a copy (as shown by a totally different databuffer address):

In [305]: a2 = a1.reshape(-1)
In [306]: a2
Out[306]: 
array([  7,  91, 205,  21,  13, 251,  56, 211,  20, 154, 164,  64],
      dtype=uint8)
In [307]: a2.view('<u4')
Out[307]: array([ 365779719, 3543726861, 1084529172], dtype=uint32)
In [308]: a2.__array_interface__['data']
Out[308]: (37940512, False)

So you can view the same databuffer with different endedness, but you can't view individual bytes in a different order without either making a non-contiguous array, or making a copy.

---

newbyteorder docs say it is equivalent to:

arr.view(arr.dtype.newbytorder(new_order))

So a.view('<u4').newbyteorder('>') is the same as a.view('<u4') . None of these changes a .