如何从变量或列表python中删除某些单词

Question

common_words = set(['je', 'tek', 'u', 'još', 'a', 'i', 'bi',
            's', 'sa', 'za', 'o', 'kojeg', 'koju', 'kojom', 'kojoj',
            'kojega', 'kojemu', 'će', 'što', 'li', 'da', 'od', 'do',
            'su', 'ali', 'nego', 'već', 'no', 'pri', 'se', 'li',
            'ili', 'ako', 'iako', 'bismo', 'koji', 'što', 'da', 'nije',
            'te', 'ovo', 'samo', 'ga', 'kako', 'će', 'dobro',
            'to', 'sam', 'sve', 'smo', 'kao'])
all = []


for (item_content, item_title, item_url, fetch_date) in cursor:
             #text = "{}".format(item_content)
             text= item_content
             text= re.sub('[,.?";:\-!@#$%^&*()]', '', text)
             text = text.lower()
             #text = [w for w in text if not w in common_words]
             all.append(text)

I want to delete certain words/stopword from either the variable "test", or later from the list "all" I put all the "text" variables from the iteration in.

I tried it like this, but this doesn't delete just words but also those letters if they exist in other words and the output is like 'd','f' for every word, and I want the format to stay the same, I just need those words in the common_words list deleted from the variable (or the list). How would I achieve that?

Answer 1

As a pythonic way for remove the punctuation from a test you can use str.translate method :

>>> "this is224$# a ths".translate(None,punctuation)
'this is224 a ths'

And for replace the words use re.sub ,first create the regex with appending the pip ( | ) to words :

reg='|'.join(common_words)
new_text=re.sub(reg,'',text)

example :

>>> s="this is224$# a ths"
>>> import re
>>> w=['this','a']
>>> boundary_words=['\b{}\b'.format(i) for i in w]
>>> reg='|'.join(oundary_words)
>>> new_text=re.sub(reg,'',s).translate(None,punctuation)
>>> new_text
' is224  ths'