删除包含熊猫数据帧同一列中值的子字符串的行的最快方法

Question

I am trying to write some efficient code that removes rows of a pandas dataframe with values in a specific column that are substrings of other values (subset of at least one value) in the same column. For example, consider column B in the following input Dataframe:

|   | A  | B          |
|---|----|------------|
| 0 | 22 | ab         |
| 1 | 33 | abc        |
| 2 | 44 | abcd       |
| 3 | 55 | a          |
| 4 | 66 | john       |
| 5 | 77 | john Doe   |
| 6 | 88 | jo         |
| 7 | 99 | john hi Doe|

Output Dataframe:

|   | A  | B          |
|---|----|------------|
| 2 | 44 | abcd       |
| 5 | 77 | john Doe   |
| 7 | 99 | john hi Doe|

Rows 0, 1, and 3 have been removed because all of their values for column B ( ab , abc , and a ) are substrings of other values in that column (ie abcd ). This is also the case for rows 4 and 6.

Answer 1

You could use some list comprehesnion to check if row strings are in other rows of the dataframe:

m = df['B'].apply(lambda x: any([x for y in df['B'] if x != y if x in y]))
df = df[~m]
df
Out[1]: 
    A            B
2  44         abcd
5  77     john Doe
7  99  john hi Doe