字符串中显示的字符，如何使用 python 删除它们

Question

I have two string that look the same but aren't.

$ more /tmp/1
'[FORM-15801]

$ more /tmp/2
'[FORM‑15801]

I see the characters here.

$ sed -n l /tmp/1
'[FORM-15801]$


$ sed -n l /tmp/2
'[FORM\342\200\22115801]$

In python, how can i convert the contents of /tmp/2 to look like /tmp/2?

Answer 1

You could use the unidecode module.

From the PyPI page:

What Unidecode provides is a middle road: the function unidecode() takes Unicode data and tries to represent it in ASCII characters (ie, the universally displayable characters between 0x00 and 0x7F), where the compromises taken when mapping between two character sets are chosen to be near what a human with a US keyboard would choose.