在python中将字节写入文件

Question

I am writing bittorrent client. It can download pieces from peers, but I can't make it write pieces to files correctly. The problem is the encoding. Because of the wrong encoding client is writing wrong bytes to file. I have found encoding called "unicode_internal". It seems to be correct one but the problem didn't go away. Despite the constant piece size(16384 bytes) sometimes the file size increases by 16386 or so. Here's how I write pieces to file. Nothing special.

with open(path, 'a', encoding='unicode_internal') as f:
    f.seek(offset, 0)
    f.write(data.decode('unicode_internal'))

I tryed to open file in 'rb' mode but it doesn't help. Part of the stdout from working client:

piece size: 16384
sum of pieces lengths: 49152
filesize: 49152

piece size: 16384
sum of pieces lengths: 65536
filesize: 65536

piece size: 16384
sum of pieces lengths: 81920
filesize: 81922 #Here it is. Size increased by 16386 bytes. The piece size is 16384

piece size: 16384
sum of pieces lengths: 98304
filesize: 98306

What am I doing wrong?

Answer 1

You need to open file in binary mode at write bytes :

data = bytes(...) # some data in bytes type
with open(path, 'ab') as f:
    f.seek(offset, 0)
    f.write(data)

When opening in text mode, independently of used encoding, Python can do transformations with line-ending. Eg on Windows it will convert single line-feed character \\n ( 0x0A ) to "Windows-style line-ending": \\r\\n ( 0x0D , 0x0A ) — two characters.