Sending bytes in a string message and changing it back to string

Question

I am coding a P2P framework in python, it works just fine and I built a filesharing app with it. Only problem, is I can't figure out how to send binary type files (zip archive in this case), in the following code you can see in a compact way what I am doing through my framework when handling files.

f = open('hhh.zip', 'rb')
filedata = f.read()
f.close()

msg = 'REPLY 5 %s' % filedata
msg = msg.encode('utf-8')

# SENDING

# ...

# RECEIVING

msg = msg.decode('utf-8')
msgtype, msglen, msgdata = msg.split(maxsplit=2)

f = open('received_archive.zip', 'xb')
f.write(msgdata)
f.close()

I am using a message structure where I indicate the message type, the lenght of the message data and the actual data. That means I can't just send the bytes of the zip file, that would obviously work. What happens is that I get my data (the bytes of archive.zip) back as a string and I can't figure out how to convert it back to bytes so I can write the received file.

The error I get is the following:

Traceback (most recent call last):
  File "test.py", line 18, in <module>
    f.write(msgdata)
TypeError: a bytes-like object is required, not 'str'

The filedata is the following: b'PK\\x03\\x04\\n\\x00\\x00\\x00\\x00\\x00z\\xbc7K\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x03\\x00\\x1c\\x00hhhUT\\t\\x00\\x03\\xb8\\xd3\\xc6Y\\xb8\\xd3\\xc6Yux\\x0b\\x00\\x01\\x04\\xe8\\x03\\x00\\x00\\x04\\xe8\\x03\\x00\\x00PK\\x01\\x02\\x1e\\x03\\n\\x00\\x00\\x00\\x00\\x00z\\xbc7K\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x03\\x00\\x18\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\xb4\\x81\\x00\\x00\\x00\\x00hhhUT\\x05\\x00\\x03\\xb8\\xd3\\xc6Yux\\x0b\\x00\\x01\\x04\\xe8\\x03\\x00\\x00\\x04\\xe8\\x03\\x00\\x00PK\\x05\\x06\\x00\\x00\\x00\\x00\\x01\\x00\\x01\\x00I\\x00\\x00\\x00=\\x00\\x00\\x00\\x00\\x00'

The msgdata after decoding it ( bytes(msgdata, 'utf-8') ): b"b'PK\\\\x03\\\\x04\\\\n\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00z\\\\xbc7K\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x03\\\\x00\\\\x1c\\\\x00hhhUT\\\\t\\\\x00\\\\x03\\\\xb8\\\\xd3\\\\xc6Y\\\\xb8\\\\xd3\\\\xc6Yux\\\\x0b\\\\x00\\\\x01\\\\x04\\\\xe8\\\\x03\\\\x00\\\\x00\\\\x04\\\\xe8\\\\x03\\\\x00\\\\x00PK\\\\x01\\\\x02\\\\x1e\\\\x03\\\\n\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00z\\\\xbc7K\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x03\\\\x00\\\\x18\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00\\\\xb4\\\\x81\\\\x00\\\\x00\\\\x00\\\\x00hhhUT\\\\x05\\\\x00\\\\x03\\\\xb8\\\\xd3\\\\xc6Yux\\\\x0b\\\\x00\\\\x01\\\\x04\\\\xe8\\\\x03\\\\x00\\\\x00\\\\x04\\\\xe8\\\\x03\\\\x00\\\\x00PK\\\\x05\\\\x06\\\\x00\\\\x00\\\\x00\\\\x00\\\\x01\\\\x00\\\\x01\\\\x00I\\\\x00\\\\x00\\\\x00=\\\\x00\\\\x00\\\\x00\\\\x00\\\\x00'"

And without decoding msgdata I am getting the first thing, but as a string, when trying to convert it back to bytes I am getting the second one.

Answer 1

bytes and str representations of the same file are pretty much equivalent, with bytes representation being able to contain slightly more things than str , including non-unicode data.

There's no need to distinguish between file types if you want to send them over a network. You can just treat everything as bytes , read them in binary mode, send them as bytes (actually python 3 won't let anything else) and write them in binary mode - no data will be lost.

with open('hhh.zip', 'rb') as f:
    filedata = f.read()

msg = b'REPLY 5 %s' % filedata  # or equivalent if they still don't support % for bytes

# SENDING
# ...
# RECEIVING

msgtype, msglen, msgdata = msg.split(maxsplit=2)

with open('received_archive.zip', 'xb') as f:
    f.write(msgdata)

Moreover, it's kind of against logic to convert to unicode ( msg.encode('utf-8') ) something that is not a human-readable data at all.