如何删除英语和西班牙语停用词

Question

I am trying to delete stop words for english and spanish. My code is working for English but not Spanish:

stopword = nltk.corpus.stopwords.words('english', 'spanish')

def remove_stopwords(text):
    text = [word for word in text if word not in stopword]
    return text
    
df['Tweet_nonstop'] = df['Tweet_tokenized'].apply(lambda x: remove_stopwords(x))

Can someone help with this problem? Thank you

Answer 1

To get English and Spanish stopwords, you can use this:

stopword_en = nltk.corpus.stopwords.words('english')
stopword_es = nltk.corpus.stopwords.words('spanish')
stopword = stopword_en + stopword_es

The second argument to nltk.corpus.stopwords.words , from the help, isn't another language:

>>> help(nltk.corpus.stopwords.words)
Help on method words in module nltk.corpus.reader.wordlist:

words(fileids=None, ignore_lines_startswith='\n') method of nltk.corpus.reader.wordlist.WordListCorpusReader instance

The first argument, fileids , can take multiple values, so, a call such as nltk.corpus.stopwords.words(fileids=('english', 'spanish')) also works as intended.

Answer 2

in addittion to the answer above, try

stopwords.words(['english','spanish'])