从熊猫数据框中删除特定字符

Question

I have a csv file which seems to have several values which have junk data that look like: ‡_¤Ëçéè_Â…

I have imported the file into a pandas dataframe. How can I get rid of these characters? I would like to delete the contents of the cell which have such characters and put in a flag value instead (something like -99999). The table has mixed data types.

import pandas as pd
import codecs
import unicodedata
import csv
import StringIO

testData = pd.read_csv('Data.csv', encoding="iso-8859-1", engine='python')

/ Using encoding utf-8 gives me an error about invalid start byte, using default engine doesn't work either. /

Any suggestions?

Answer 1

IF you know what characters you are willing to accept, you could use a regex to filter your values, something like:

testData['stringcol'].where(testData['stringcol'].str.contains('[^A-Za-z0-9\s]'), 
-999999)