Stemmer function 接受一个字符串并返回列表中每个单词的词干

Question

I am trying to create this function which takes a string as input and returns a list containing the stem of each word in the string. The problem is, that using a nested for loop, the words in the string are appended multiple times in the list. Is there a way to avoid this?

def stemmer(text):
    
    stemmed_string = []
    res = text.split()
    suffixes = ('ed', 'ly', 'ing')
    
    for word in res:
            for i in range(len(suffixes)):
                if word.endswith(suffixes[i]):
                    stemmed_string.append(word[:-len(suffixes[i])])
                elif len(word) > 8:
                    stemmed_string.append(word[:8])
                else:
                    stemmed_string.append(word)
    
    return stemmed_string

If I call the function on this text ('I have a dog is barking') this is the output:

['I',
 'I',
 'I',
 'have',
 'have',
 'have',
 'a',
 'a',
 'a',
 'dog',
 'dog',
 'dog',
 'that',
 'that',
 'that',
 'is',
 'is',
 'is',
 'barking',
 'barking',
 'bark']

Answer 1

You are appending something in each round of the loop over suffixes. To avoid the problem, don't do that.

It's not clear if you want to add the shortest possible string out of a set of candidates, or how to handle stacked suffixes. Here's a version which always strips as much as possible.

def stemmer(text):
    stemmed_string = []
    suffixes = ('ed', 'ly', 'ing')
    
    for word in text.split():
        for suffix in suffixes:
            if word.endswith(suffix):
                word = word[:-len(suffix)]
        stemmed_string.append(word)
    
    return stemmed_string

Notice the fixed syntax for looping over a list, too.

This will reduce "sparingly" to "spar", etc. Like every naïve stemmer, this will also do stupid things with words like "sly" and "thing".

Demo: https://ideone.com/a7FqBp