删除特殊字符python数据框

Question

I wanted to remove special characters from a column and some words I choose.

df['tweet_text'][0]
'\\": \\"#\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 TEXAS Corona update 19-MAY-21\\\\n\\\\nTotal Deaths 51","180\\\\n\\\\nhttps://t.co/jeoAqC07Oq\\\\n\\\\n#\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588updates #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588 #\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\\\u2588\\"","\\"'

I used

df['tweet_text'] = df['tweet_text'].str.replace('[#,@,&,{,},",:,//,\\\n,-,\\\\,u2588]', '')

' TEXAS Corona pdate 19MAY1nnTotal Deaths 110nnhttpst.cojeoAqC07Oqnn pdates '

As you can see in the out put, there "nn" not removed, and every "u" is removed . Can you help me figure this out? thank you!

Answer 1

.replace() uses regular expressions. Your regex character class '[#,@,&,{,},",:,//,\\\\\\n,-,\\\\\\\\,u2588]' is parsed as

[#,@,&,{,},",:,//,\
,-,\\,u2588]

so it will match the newline character and the characters "#&,/258:@\\u{}\u003c/code> (not a dash, though, since it's a range delimiter in regexps).

You'll need to read up on the syntax for regular expressions.

(However, if your dataframe has a string like that to begin with, I'm afraid your data is broken in other ways too...)