How do I print unicode characters in Python 3 CGI?

Question

I am trying to run a python CGI script that has some Unicode strings. It works fine in console but when I run it in the browser it gives following error-

UnicodeEncodeError: 'ascii' codec can't encode character '\…' in position 198: ordinal not in range(128)

I have tried these two -
encode('ascii','ignore') unicodedata.normalize('NFKD',var).encode('ascii', 'ignore').strip()
var is the variable that holds the unicode string.

Now how do I print Unicode or ascii without error in the browser ?

Answer 1

When Python prints Unicode strings to the console it usually detects the console encoding and automatically encodes the Unicode strings using that encoding. For CGI there is no terminal so the default is ascii . You can change stdout to another encoding by using:

import codecs
sys.stdout = codecs.getwriter('utf8')(sys.stdout.buffer)

You can use a similar line for stderr as well.