应用 function 不适用于数据框列

Question

I am trying to remove special characters like ",",".","-"(except comma) from the "Actors" column of my pandas data-frame. For this I use the apply method on the "Actors" column

df['Actors']= df['Actors'].apply(lambda x : x.lower().replace("[^a-zA-Z,]","",)
df['Actors'].head()

The output of the above snippet is shown below and we can see no special characters have been replaced:

1    tim robbins, morgan freeman, bob gunton, willi...
2    marlon brando, al pacino, james caan, richard ...
3    al pacino, robert duvall, diane keaton, robert...
4    christian bale, heath ledger, aaron eckhart, m...
5    martin balsam, john fiedler, lee j. cobb, e.g....
Name: Actors, dtype: object

But when I try resolving the above issue using the snippet below, the code works:

df['Actors'] = df['Actors'].str.lower().str.replace("[^a-zA-Z,]","")
df['Actors'].head()

1    timrobbins,morganfreeman,bobgunton,williamsadler
2    marlonbrando,alpacino,jamescaan,richardscastel...
3    alpacino,robertduvall,dianekeaton,robertdeniro
4    christianbale,heathledger,aaroneckhart,michael...
5    martinbalsam,johnfiedler,leejcobb,egmarshall
Name: Actors, dtype: object

I want to know what is it with the apply function that it doesn't work properly while replacing characters?

Answer 1

You call apply on series, so x in the lambda is a single string of each row of the series. So, x.lower().replace is python replace . Python replace doesn't support regex. so it considers "[^a-zA-Z,]" as a whole string and it looks for that substring in each x . It couldn't find it so nothing got replaced.

On the other hand, Pandas str.replace default option is regex=True , so it considers "[^a-zA-Z,]" as a regex pattern and replaces everything properly

Answer 2

It does not work because you do a replace on a string, formally you do str.replace("[^a-zA-Z,]","",) . Your sting do not contain those characters [^a-zA-Z,] so nothing is removed. If you prefer, python do interpret those characters as regex, but simply as string elements.

To work you should do it like this, it's just to answer your question because the preferred way to do it is with your second exemple.

remove = re.compile(r"[^a-zA-Z,]")
df['Actors']= df['Actors'].apply(lambda x : re.sub(remove, "", x.lower()))

Herw are some documentation:

• python str replace
• pandas str replace