用于删除奇怪字符的Python正则表达式

Question

Hello,

I have a list of string with some strange characters (like: ). For instance:

'Replay fortement conseillé �\x9f\x98\x82�\x9f\x98\x82'

Or:

'Le papa du mois �\x9f\x91\x8a'

I want to remove \\x9f\\x91\\x8a and \\x9f\\x98\\x82 \\x9f\\x98\\x82 from these strings.

I tried this regex: ((.?)\\\\x[0-9]([az]|[0-9])(.?)+)+ but it doesn't work. I'm a newbie in regex so I ask for help.

Thanks you

Answer 1

It's probably better to handle those characters instead of removing them, but if you want to remove them in Python you can do that without regular expressions.

text.decode("ascii", "ignore")

This line will decode a byte array in Python and only keep ASCII characters.

Answer 2

To hold specific characters in the string like é in conseillé .

You should find the substring that you want to delete And to do this, you need to find the beginning and the end of the substring.

This is done better with stringed methods

for an example:

if in any string start character for delete is :

and end of string is len of the string:

re.sub(r' .*','', 'Replay fortement conseillé \\x9f\\x98\\x82 \\x9f\\x98\\x82')

i hope this could help you

Answer 3

In my experience it's a little safer to create a list of 'safe' characters to keep. What you are looking to do today is 'fix' that sentence and get rid of yucky stuff. But what if some more goofball stuff shows up? I have a requirement for the data I process to only keep 'standard ascii' as decided by a business owner, so I use this regex:

text = re.sub("[^\x20-\x7E]", "", text)

That way I remove anything that isn't in that character class, pretty much anything not on a standard keyboard. You may have better luck going this route. It's hard to predict what trash characters are going to come down the road, and then you end up editing your regex to keep adding stuff to strip out. Make a list of stuff to keep :)