Python Socket utf-8 解码

Question

I want to get the response of a GET request into a string, and i came up with the following code:

 import socket target_host = "www.google.com" target_port = 80 # create a socket object s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # connect the client s.connect((target_host, target_port)) # send some data request = "GET / HTTP/1.1\\r\\nHost:%s\\r\\n\\r\\n" % target_host s.send(request.encode("utf-8")) full_msg = "" # Prevent recv() function to stop the script to wait until it receives more data, even if there is no more. s.settimeout(1) flag = True while flag: # receive some data try: response = s.recv(4096) full_msg = full_msg + str(response) print("Adding msg") except Exception as e: print(full_msg) flag = False print(e) print("Loop ended") print(type(full_msg))

The thing is, when i try do decode the response in s.recv(4096) replacing it with this code:

 response = s.recv(4096).decode("utf-8")

I get the following exception:

 'utf-8' codec can't decode byte 0xe8 in position 1025: invalid continuation byte

I don't know how to fix this as i can't modify the characters i get from the response and i need to delete the " b' " character which messes up my full_msg string if i don't decode it each time in the loop.

Also, in the docs it says that the .recv() method returns a string, but i seem to be getting a byte-like object. Any idea is welcomed and i am also open to know any ways my code could be improved.

Answer 1

Per the data in your response, Content-Type: text/html; charset=ISO-8859-1 Content-Type: text/html; charset=ISO-8859-1 . The response is not UTF-8. .decode('iso-8859-1') instead. Better yet use requests :

import requests
r = requests.get('https://www.google.com')
print(r.status_code)
print(r.headers['content-type'])
print(r.encoding)
# print(r.text)    # already decoded with r.encoding
# print(r.content) # byte string of original encoded data
# print(r.json())  # If the response is JSON data, This loads the data.