如果字符串与列表中的字符串匹配，我如何从句子中删除字符串

Question

I have a pandas.Series with sentences like this:

0    mi sobrino carlos bajó conmigo el lunes       
1    juan antonio es un tio guay                   
2    voy al cine con ramón                         
3    pepe el panadero siempre se porta bien conmigo
4    martha me hace feliz todos los días 

on the other hand, I have a list of names and surnames like this:

l = ['juan', 'antonio', 'esther', 'josefa', 'mariano', 'cristina', 'carlos']

I want to match sentences from the series to the names in the list. The real data is much much bigger than this examples, so I thought that element-wise comparison between the series and the list was not going to be efficient, so I created a big string containing all the strings in the name list like this:

'|'.join(l)

I tried to create a boolean mask that later allows me to index the sentences that contains the names in the name list by true or false value like this:

series.apply(lambda x: x in '|'.join(l))

but it returns:

0    False
1    False
2    False
3    False
4    False

which is clearly not ok.

I also tried using str.contains() but it doesn't behave as I expect, because this method will look if any substring in the series is present in the name list, and this is not what I need (ie I need an exact match).

Could you please point me in the right direction here?

Thank you very much in advance

Answer 1

If need exact match you can use word boundaries:

series.str.contains('|'.join(rf"\b{x}\b" for x in l))

For remove values by list is use generator comprehension with filtering only non matched values by splitted text:

series = series.apply(lambda x: ' '.join(y for y in x.split() if y not in l))
print (series)
                            
0                mi sobrino bajó conmigo el lunes
1                                  es un tio guay
2                           voy al cine con ramón
3  pepe el panadero siempre se porta bien conmigo
4             martha me hace feliz todos los días

Answer 2

import re

data = ["mi sobrino carlos bajó conmigo el lunes", "juan antonio es un tio guay", "martha me hace feliz todos los días"]

regexs = ['juan', 'antonio', 'esther', 'josefa', 'mariano', 'cristina', 'carlos']

for regex in regexs:

    for sentence in data:

        if re.match(regex, sentence):

            print True
        
        else:

            print False

I guess something simple like that could work

cf: https://docs.python.org/fr/3/library/re.html

Answer 3

Regex to check if a word at the start or at the end or in between

df = pd.DataFrame({'texts': [
                             'mi sobrino carlos bajó conmigo el lunes',
                             'juan antonio es un tio guay',
                             'voy al cine con ramón',
                             'pepe el panadero siempre se porta bien conmigo',
                             'martha me hace feliz todos los días '
                             ]})

names = ['juan', 'antonio', 'esther', 'josefa', 'mariano', 'cristina', 'carlos']

pattern = "|".join([f"^{s}|{s}$|\\b{s}\\b" for s in names])

df[df.apply(lambda x: 
            x.astype(str).str.contains(pattern, flags=re.I)).any(axis=1)]

Answer 4

one option is set intersection:

l = set(['juan', 'antonio', 'esther', 'josefa', 'mariano', 'cristina', 'carlos'])
s.apply(lambda x: len(set(x.split()).intersection(l))>0)

Answer 5

For exact match. Try;

df.text.str.contains("|".join(l))

Otherwise, simply use regular expression to replace substring with '' . Call the substring using list comprehension

df.replace(regex=[x for x in l], value='')
                          

                                   text
0               mi sobrino  bajó conmigo el lunes
1                                  es un tio guay
2                           voy al cine con ramón
3  pepe el panadero siempre se porta bien conmigo
4             martha me hace feliz todos los días

Answer 6

If you want a little more flexibility for processing, you can have your custom exact_match function as below:

import re 

def exact_match(text, l=l):
    return bool(re.search('|'.join(rf'\b{x}\b' for x in l), text))

series.apply(exact_match)

Output:

0     True
1     True
2    False
3    False
4    False
dtype: bool