如何修复：UnicodeDecodeError：“utf-8”编解码器无法解码 position 中的字节 0x81 18：起始字节无效

Question

I am trying to make a remote shell, using sockets. Some commands are working like netstat or dir . But when I try to use a ping , this error message comes up:

• UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in position 18: invalid start byte

This problem only occurs on the server side, if I try to encode the message in utf-8. With a normal byte string it works fine. I have also tried to receive first and then encode the string.

Server side:

    while True:
        cmd = input(">>>")

        if len(str.encode(cmd)) > 0:
            connection.send(str.encode(cmd))

            client_response = str(connection.recv(1024), encoding = "utf-8")

            print (client_response, end = "")

Client side:

    while True:
        data = sock.recv(1024).decode("utf-8")

        proc = subprocess.Popen(shlex.split(data),
                                shell = True,
                                stdin = subprocess.PIPE,
                                stdout = subprocess.PIPE,
                                stderr = subprocess.PIPE)

        output = proc.stdout.read()+proc.stderr.read()

        sock.send(output)

Maybe someone could give me a tip.

Answer 1

As an aside, I think your client and server definitions may be the wrong way around. The client should be the one accepting user input and sending that off to the server for execution. However, that's not really relevant to your question.

I will, however, refer to them as user side (where user inputs commands) and execution side (where the process is run) to hopefully avoid misunderstanding.

---

I see that no encoding is done to the output on the execution side of the link, you simply return the standard output and error as-is.

I'd suggest printing that out before sending so you can see what it contains, as well as printing it out before attempting conversion on the user side:

# On exection side:
output = proc.stdout.read()+proc.stderr.read()
print('before send', output)

# On user side:
client_response = connection.recv(1024)
print('got response', client_response)
client_response = str(client_response, encoding = "utf-8")

The start byte 0x81 is very much invalid as a UTF-8 start byte - it should either have the leftmost bit set to zero or the leftmost two bits set to one. 0x81 is binary 1000 0001 so does not meet that requirement.