python 使用 utf-16 编码和解码

Question

>>>> enbytes = b'\xdf\x81\x9e\xbf"Q\xa37\xd0\x7f\x18\x1d:J\xe2\xa1'

>>>> enbytes.decode('utf-16').encode('utf-16')

b'\xff\xfe\xdf\x81\x9e\xbf"Q\xa37\xd0\x7f\x18\x1d:J\xe2\xa1'

why enbytes does not equal encoding bytes. How to fix that.

Answer 1

That's the byte-order mark (BOM), you can just use a specific endianessto prevent the addition of the BOM, so little endian:

>>> enbytes.decode('utf-16').encode('utf-16-le')
b'\xdf\x81\x9e\xbf"Q\xa37\xd0\x7f\x18\x1d:J\xe2\xa1'
>>> enbytes == enbytes.decode('utf-16').encode('utf-16-le')
True