ctypes bigendianstructure littlendianstructure返回单个字节不同的结果

Question

I am seeing different results when using ctypes and Structure(s) with big and little endian-ness types, and fields with specific bit lengths. I would expect the bit ordering in a byte to be the same for both endia-ness, but the results are providing different answers. The results are from a centos 6.8 64 bit distro on an intel box.

>>> import ctypes
>>>
>>> class header_struct(ctypes.Structure):
...       _fields_ = [ ('f1',ctypes.c_ubyte,4),
...                    ('f2',ctypes.c_ubyte,4) ]
...
>>> class header_struct_be(ctypes.BigEndianStructure):
...       _fields_ = [ ('f1',ctypes.c_ubyte,4),
...                    ('f2',ctypes.c_ubyte,4) ]
...
>>> class header_struct_le(ctypes.LittleEndianStructure):
...       _fields_ = [ ('f1',ctypes.c_ubyte,4),
...                    ('f2',ctypes.c_ubyte,4) ]
...
>>> a='\x0A'
>>> x=header_struct.from_buffer_copy(a)
>>> x_be=header_struct_be.from_buffer_copy(a)
>>> x_le=header_struct_le.from_buffer_copy(a)
>>>
>>> print " sizeof(x) ", ctypes.sizeof(x)
 sizeof(x)  1
>>> print " sizeof(x_be) ", ctypes.sizeof(x_be)
 sizeof(x_be)  1
>>> print " sizeof(x_le) ", ctypes.sizeof(x_le)
 sizeof(x_le)  1
>>>
>>> x.f1
10
>>> x_be.f1
0
>>> x_le.f1
10
>>>
>>>
>>> x.f2
0
>>> x_be.f2
10
>>> x_le.f2
0
>>>

Answer 1

The bit order inside a byte is not the same for both endianness - of course,in any architecture, you copy a number into a byte, you get back the same byte. And in both architectures, the least significant bit is addressed as bit "0". Since data is moved around in bytes, and this is just emulation, anyway these bit values are not actually mirrored in a x86 architecture in memory. That is valid for the wat ctypes, and likely, C code you would generate playing along with that do it.

However, if you subdivide a byte in fields, the relative position of these fields is mirrored inside the byte -

You can check that in an easier way using the ctypes.Union construct (that way you also rule out padding-bytes side effects as possible cause of the numbers you see):

import ctypes
class header_struct(ctypes.Structure):
       _fields_ = [ ('f1',ctypes.c_ubyte,4),
                  ('f2',ctypes.c_ubyte,4) ]

class big_endian(ctypes.BigEndianStructure):
      _fields_ = [ ('b{}'.format(i), ctypes.c_ubyte, 1) for i in range(8) ]
class little_endian(ctypes.LittleEndianStructure):
      _fields_ = [ ('b{}'.format(i), ctypes.c_ubyte, 1) for i in range(8) ]

class le_byte(ctypes.LittleEndianStructure):
    _fields_ = [('value', ctypes.c_ubyte)]

class be_byte(ctypes.BigEndianStructure):
    _fields_ = [('value', ctypes.c_ubyte)]

class Union(ctypes.Union):
    _fields_ = [('le', le_byte), ('be', be_byte), ('lebits', little_endian), ('bebits', big_endian)]

And on the interactive console:

In [319]: u = Union()

In [320]: u.le.value = 0x80

In [321]: u.be.value  # not mirrored
Out[321]: 128

In [322]: u.lebits.b7 
Out[322]: 1

In [323]: u.lebits.b0
Out[323]: 0

In [324]: u.bebits.b7
Out[324]: 0

In [325]: u.bebits.b0
Out[325]: 1

Seeing that you are probably working on some practical code, and not just playing along, my advice would be to keep all structures that have to handle sub-byte fields as LittleEndianStructure, and create an Union with a BigEndianStructure with the same byte-size for whenever you have to copy bytes from the buffer to do your I/O.

Putting in another way, just to be sure it is clear: all sub-byte bit maniplation is performed on the structure declared as LittleEndian. Then, to copy multi-byte data to and from this structure, you put it in a union with another structure that only has fields with an integer number of bytes - and which can be declared BigEndian - and perform all the copying of data referencing this other structure.

Also, do a lot of testing on the whole thing :-).