getting unicode error while decode packet recived on tcp socket

Question

my python client is receiving packets from a c++ TCP socket which is acting as a server but when I try to decode the I am getting UnicodeDecodeError:'utf-8' codec can't decode byte 0xeb in position 2: invalid continuation byte.

this is the packet: b'\x19\x00\xeb\x03NIFTY21JANFUT|1003.25\x19\x00\xeb\x03NIFTY21JANFUT|1003.25'

packet = b'\x19\x00\xeb\x03NIFTY21JANFUT|1003.25\x19\x00\xeb\x03NIFTY21JANFUT|1003.25'
print(packet)
print()

for i in packet:
    print(i)

packet.decode()

click here for image 1

click here for image 2

Answer 1

You should send data in utf-8 encoding in c++ side, Unfortunately, I don't know how to set it correctly, but you can use chardet to detect encoding and decode packet respectfully

import chardet
packet_encoding = chardet.detect(packet)["encoding"]
decoded_packet = packet.decode(packet_encoding)