how can i decode unicode string in python?

Question

Wikipedia API encodes string into unicode format

"Golden Globe Award for Best Motion Picture \u2013 Drama"

how can i convert it back to

"Golden Globe Award for Best Motion Picture – Drama"

Answer 1

The Wikipedia API returns JSON data, use the json module to decode:

json.loads(inputstring)

Demo:

>>> import json
>>> print json.loads('"Golden Globe Award for Best Motion Picture \u2013 Drama"')
Golden Globe Award for Best Motion Picture – Drama

If you instead have a string that starts with u'' , you already have a Python unicode value and are looking at the representation of that string:

>>> json.loads('"Golden Globe Award for Best Motion Picture \u2013 Drama"')
u'Golden Globe Award for Best Motion Picture \u2013 Drama'

Just print that value to have Python encode it to your terminal codec and represent that em-dash character in a format your terminal will understand.

You may want to read up about Python and Unicode and encodings before you continue, if you do not understand what the difference is between a unicode value and byte strings:

• The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky

• The Python Unicode HOWTO

• Pragmatic Unicode by Ned Batchelder