从不在列表中的每行数据框中的列中删除字符串

Question

Say I have a list of words:

listOfWords = ['Apple','Orange','Banana','Potato']

And my data frame looks like this:

In:

ColumnA:
['Apple','Turnip','Banana','Potato']
['Apple','Orange','Banana','Potato']
['Apple','Orange','Pastry','Potato']
['Melon','Orange','Banana','Potato']
['Apple','Orange','Banana','Sandwich']

I am currently running the following code to retrieve the desired output

for index, row in df.iterrows():
    for word in df['Column']:
        if word not in listOfWords:
            word.replace(word,"")



Out:

ColumnA:
    ['Apple','Banana','Potato']
    ['Apple','Orange','Banana','Potato']
    ['Apple','Orange','Potato']
    ['Orange','Banana','Potato']
    ['Apple','Orange','Banana']

I am currently running this on 12,000 records and a list of length 12,000. It has been running without errors for a few hours, however I am unsure if this is the most efficient way to do this.

Answer 1

Use list comprehension in apply or nested list comprehension :

df['ColumnA']= df['ColumnA'].apply(lambda x: [y for y in x if y in listOfWords]) 
#another solution
#df['ColumnA'] = [[y for y in x if y in listOfWords] for x in df['ColumnA']]
print (df)
                           ColumnA
0          [Apple, Banana, Potato]
1  [Apple, Orange, Banana, Potato]
2          [Apple, Orange, Potato]
3         [Orange, Banana, Potato]
4          [Apple, Orange, Banana]

Or if order is not importat use set s with intersection:

s = set(listOfWords)
df['ColumnA']= df['ColumnA'].apply(lambda x: list(set(x) & s))
print (df)
                           ColumnA
0          [Banana, Potato, Apple]
1  [Banana, Potato, Orange, Apple]
2          [Potato, Orange, Apple]
3         [Banana, Potato, Orange]
4          [Banana, Orange, Apple]