JSON 文件如何删除不需要的字符

Question

so I scraped some data into a JSON file format but there are some unwanted characters in the saved data for example:

"quote_text": "\“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.\”", "author": "Albert Einstein", "tags": [ "change", "deep-thoughts", "thinking", "world"

So how can I remove these \“ type characters from the file in python

Answer 1

Replace method:

If you have only 1 or 2 characters to remove I suggest that you use the string .replace() method:

An example can be on the quote_text key

your_dict['quote_text'].replace('\u201c','')

Regex:

If you are struggling with multiple characters I suggest you dive into Regex

More:

If you wish to apply your function to the entire dictionnary values you can use dict comprehensions:

d2 = dict((k, f(v)) for k, v in d1.items())

d1 being your original dictionnary and f your function.

In our example it would be:

d2 = dict((k, v.replace('\u201c','')) for k, v in d1.items())

Answer 2

If you want to remove multiple characters you can use a list to indicate what letters to remove:

text = '{ "work": "\u201cfun\u201c", "foo": ["bar", "baz"] }'
remove_chars = ['u201c', 'b', 'f']
new_text = ''.join([ch for ch in text if ch not in remove_chars])

To replace unwanted characters make a dictionary to hold the substitutions then make the changes:

subs = {
  '\u201c': "'",
  'z': 't'
}
text = '{ "work": "\u201cfun\u201c", "foo": ["bar", "baz"] }'
letter_list = [(subs[ch] if ch in subs else ch)  for ch in text]
new_text = ''.join(letter_list)

Answer 3

Let's assume dictionary as d. As I can see, there are different unicode characters like \“ , \” . If you want to remove all Unicode characters at once, you can do something like this:

one liner code:

d['quote_text'].encode("ascii", "ignore").decode('utf-8')

Explanation in detail:

The below one line code remove all the unicode characters and will return value in bytes.

remov_unicode_char = d['quote_text'].encode("ascii", "ignore")

Now, in order to convert into string, you can decode it.

convert_str =  remov_unicode_char.decode("utf-8")

Now, you can check the result by printing it.

print(convert_str)

output:

The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.