根据另一个列表从pandas dataframe列中的列表中删除值

Question

I have a column in a dataframe which contains lists. I want to be able to remove elements from these lists based on elements that I have in another list (as shown below).

I tried to use list comprehension but it seems to give no result.

import pandas as pd

sys_list = ['sys1', 'sys2', 'sys3']
df = pd.DataFrame({'A':[['sys1', 'sys2', 'user1'], 
                        ['user3', 'user6', 'user1'], 
                        ['sys1', 'sys2', 'sys3']]})

df['A'] = [item for item in df['A'] if item not in sys_list]

print(df)

                       A
0    [sys1, sys2, user1]
1  [user3, user6, user1]
2     [sys1, sys2, sys3]

I need to achieve this:

                       A
0                [user1]
1  [user3, user6, user1]
2                     []

Any thoughts?

Answer 1

with apply :

df.A.apply(lambda x: [i for i in x if i not in sys_list])

---

0                  [user1]
1    [user3, user6, user1]
2                       []
Name: A, dtype: object

Answer 2

Use Series.apply :

df['B'] = df['A'].apply(lambda x: [item for item in x if item not in set(sys_list)])
print (df)
                       A                      B
0    [sys1, sys2, user1]                [user1]
1  [user3, user6, user1]  [user3, user6, user1]
2     [sys1, sys2, sys3]                     []

Or similar list comprehension like deleted answer:

df['B'] = [[item for item in l if item not in set(sys_list)] for l in df['A']]

Or solution with set s with set.difference :

df['B'] = df['A'].map(set(sys_list).difference).map(list)

Answer 3

You may use sets for a better performance (this approach assumes that the order within the lists is not important, as it will change):

sys_set = set(['sys1', 'sys2', 'sys3'])

df['A'] = (df.A.map(set)-sys_set).map(list)

---

print(df)
                    A
0                [user1]
1  [user6, user1, user3]
2                     []