Python在html电子邮件附件中解码法语字符

Question

I'm trying to decode an html attachment file of an email that I take on an IMAP Server. If the html file contain normal character it's working without problem, but when you have some french é character I have this: "vous a \\xc3\\xa9t\\xc3\\xa9 envoy\\xc3\\xa9e par" I also have all \\n \\r that appear.

I use beautifulsoup to make some search on the html code. I also use a loop to check all the mail(Not present in this code)

imap_server = imaplib.IMAP4_SSL("server",993)
imap_server.login(username, password)
imap_server.select("test")
result, data = imap_server.uid('search', None, "UnSeen")
latest_email_uid = data[0].split()[-1]
result, data = imap_server.uid('fetch', latest_email_uid, '(RFC822)')
raw_email = data[0][1]
raw_email=str(raw_email, 'UTF8')
msg = email.message_from_string(raw_email)

I walk in the mail, if I find some html I decode it from base64 and send to beautifulsoup. After that I print it with utf-8 conversion. If I replace encode.('utf-8') by latin-1 I have also special char.

if msg.is_multipart(): 
    for part in msg.walk():
        if part.get_content_type() == 'text/html':
            attachment= (part.get_payload(decode=1))
            soup=BeautifulSoup(attachment)
            print (soup.prettify().encode('utf-8'))
        else:
            print ("No HTML")

I tried to encode,decode in a lot a charset without having something nice. I have also tried with base64.b64decode(text).decode('utf-16') but still have the same \\xc3\\xa9

Answer 1

You see the special characters because you are encoding to UTF-8 or Latin-1:

>>> print('\xe9')
é
>>> print('\xe9'.encode('utf8'))
b'\xc3\xa9'
>>> print('\xe9'.encode('latin1'))
b'\xe9'
>>> print('Hello world!\n'.encode('utf8'))
b'Hello world!\n'

When printing a bytes literal, Python shows the repr() representation of the value, which replaces any byte that does not represent a printable ASCII codepoint with the \\x.. escape sequence; some are replaced with the shorter two-character escapes, such as \\r and \\n . This makes the representation both re-usable as a Python bytes literal and more easily logged to files and terminals not set up for international character sets.

print() handles encoding for you. Just print the .prettify() output directly .

If printing Unicode to your terminal or console does not work, and instead raises a UnicodeDecodeError , your terminal or console is not configured to handle Unicode text properly. Consult the PrintFail Python Wiki page to troubleshoot.