UTF-8和彩色打印问题

Question

I have a console program that outputs in wonderful colour. For errors, the following code is used with some trivial examples at the bottom.

# coding: utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from sys import stderr
from colored import fg
from colored import attr
from locale import getpreferredencoding

def format_error(x):
    return '{0}{1}{2}'.format(fg(88), x, attr('reset'))

def print_error(x):
    msg = format_error('✗  {0}\n'.format(x))
    stderr.write(msg.encode(getpreferredencoding()))

print_error(str('ook'))
print_error(unicode(b'café', 'UTF-8'))

I have no control over that x is. It could be anything. Also, some of this script is called from a GUI that captures stdout / stderr via glib-spawn-async . As such, from time to time, I get UnicodeDecodeError errors. I have read the Unicode HOWTo but clearly I am missing something.

How can I harden my code such that UnicodeDecodeError are never raised?

For example, within a gtk.textview , I get the following whereas on the console, all is fine. Trace has been cut to remove irrelevant data.

 File "/home/usr/nifty_logger.py", line 96, in print_success
    sys.stdout.write(msg.encode(getpreferredencoding()))
  File "/home/usr/.virtualenvs/rprs_bootstrap/lib64/python2.7/codecs.py", line 351, in write
    data, consumed = self.encode(object, self.errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

Answer 1

The encode() takes an optional argument defining the error handling:

str.encode([encoding[, errors]])

From the docs:

Return an encoded version of the string. Default encoding is the current default string encoding. errors may be given to set a different error handling scheme. The default for errors is 'strict', meaning that encoding errors raise a UnicodeError. Other possible values are 'ignore', 'replace', 'xmlcharrefreplace', 'backslashreplace' and any other name registered via codecs.register_error(), see section Codec Base Classes. For a list of possible encodings, see section Standard Encodings.

In your case:

msg.encode(getpreferredencoding(), 'backslashreplace')