如何从python中的Dictionary数据中删除unicode字符

Question

After using request library , am getting below dict in response.json()

{u'xyz': {u'key1': None, u'key2': u'Value2'}}

I want to remove all unicode characters and print only key value pairs without unicode chars

I have tried below method to remove , but it shows malformed string

>>> import json, ast
>>> c = {u'xyz': {u'key1': None,u'key2': u'Value2'}}
>>> ast.literal_eval(json.dumps(c))

Getting 'ValueError: malformed string'

Any suggestion on how to do it ?

Answer 1

Change your None to 'None':

 c = {u'xyz': {u'key1': 'None', u'key2': u'Value2'}}

it is a casting issue - ast likes str's

Also, maybe u want to change all None to empty str or 'None' str... See this thread : Python: most idiomatic way to convert None to empty string? with this code, i've changes the empty string to 'None':

def xstr(s):
    if s is None:
        return 'None'
    return str(s)

Answer 2

This snippet will helps you to preserve the data without unicode prefix notation u :

>>> import json
>>> c = {u'xyz': {u'key1': u'Value1',u'key2': u'Value2'}}
>>> print c
{u'xyz': {u'key2': u'Value2', u'key1': u'Value1'}}
>>> d = eval(json.dumps(c))
>>> print d
{'xyz': {'key2': 'Value2', 'key1': 'Value1'}}

json.dumps() will convert the dict to string type and eval() will reverse it.

Note: key1 value has changed from None to 'value1' for testing purpose

Answer 3

You can use unicodestring.encode("ascii","replace")

>>> ustr=u'apple'
>>> ustr
u'apple'
>>> astr=ustr.encode("ascii","replace")
>>> astr
'apple'

Answer 4

I don't really understand why you want this. Your variable is a normal Python dict with normal Unicode strings, and they happen to be printed as u'' to distinguish them from bytestrings, but that shouldn't matter for using them.

If you want to save them as strings to read them as data later, JSON is a fine format for that. So no need to call request's .json() function at all, just use the response's .text attribute -- it's already JSON, after all.

Your try

>>> ast.literal_eval(json.dumps(c))

Fails because you first turn c into JSON again, and then try to parse it as a Python literal. That doesn't work because Python isn't JSON; in particular one has null and the other has None .

So maybe you want to change the Unicode strings into bytestrings? Like by encoding them as UTF8, that might work:

def to_utf8(d):
    if type(d) is dict:
        result = {}
        for key, value in d.items():
            result[to_utf8(key)] = to_utf8(value)
    elif type(d) is unicode:
        return d.encode('utf8')
    else:
        return d

Or something like that, but I don't know why you would need it.