试图从列表中删除匹配的字符串

Question

I have two strings one that contains a sentence and another a list of names. Look to the commets in the code to see how they are formatted.

I am trying to go through a column in a database and remove all the names from the sentence.

The sentences appear to be unchanged after calling the function.

with open('names.txt', 'r') as f:
    NAMES = set(f.read().splitlines())
NAMES = [name.lower() for name in NAMES]

def remove_names(df, col, NAMES):
    for idx in range(df.shape[0]):
        print("\r", idx, df.shape[0], idx/df.shape[0], end="\r")
        # your list of texts
        texts=df[col][idx]
        #texts looks like
        #['explain', 'decided', 'make', 'coverage', 'area', 'rubbish', 'online', 'checker', 'correct', 'sky', 'account', 'connection']
        holder_list = []
        for word in texts:
            #NAMES looks like
            # ['pascha', 'lang', 'desaray', 'camielle', 'marquasha', 'trasha', 'shaquila',...
            for name in NAMES:
                if name == word or name == word + "'s":
                    continue
                else:
                    holder_list.append(word)
        df[col][idx] = holder_list.copy()
    return df[col]
df_norm['Full Text'] = remove_names(df_norm, 'Full Text', NAMES)

Answer 1

I updated your remove_names function:

def remove_names(df_list, NAMES):
    new_list = [x for x in df_list if x not in NAMES]
    return new_list


df_norm['Full Text'] = df_norm['Full Text'].apply(remove_names, args = ([NAMES]))

print(df_norm)

If you want to get rid of the remove_names function altogether, you can use a lambda function, which updates the column using one line of code instead:

df_norm['Full Text'] = df_norm['Full Text'].apply(lambda df_list: [x for x in df_list if x not in NAMES])

Note:

The code above assumes that your df_norm['Full Text'] column looks something like this:

Answer 2

Since you repeatedly need to test for membership of a word to NAMES , you should make NAMES a set rather than a list. Testing membership in sets is much faster than testing membership in lists.

You can use pandas' apply to apply a function to every row of a dataframe.

If a row of your dataframe is a list of words, you can implement the function to apply to every row like this:

def remove_names(list_of_words, set_of_names):
    return [word for word in list_of_words if word not in set_of_names]

# TEST:
print( remove_names(['Alice', 'gives', 'Bob', 'an', 'apple'], {'Alice', 'Bob'}) )
# ['gives', 'an', 'apple']

If a row of your dataframe is a sentence, ie a single string with space-separated words, you can implement the function to apply to every row like this:

def remove_names(sentence, set_of_names):
    return ' '.join(word for word in sentence.split() if word not in set_of_names)

# TEST:
print( remove_names('Alice gives Bob an apple', {'Alice', 'Bob'}) )
# 'gives an apple'

And then apply it to a column of the dataframe:

import pandas as pd

df = pd.DataFrame({'id':[47, 28], 'sentence': ['Alice gives Bob an apple', 'An apple is given to Alice']})
df['nonames'] = df['sentence'].apply(remove_names, args=({'Alice', 'Bob'},))

print(df)
#    id                    sentence               nonames
# 0  47    Alice gives Bob an apple        gives an apple
# 1  28  An apple is given to Alice  An apple is given to