类型错误：字符串索引必须是整数 - 清理我的文本

Question

Trying to clean the tweets with this function:

class PreProcessTweets:
    def __init__(self):
        self._stopwords = set(stopwords.words('english') + list(punctuation) + ['AT_USER','URL'])

    def processTweets(self, list_of_tweets):
        processedTweets=[]
        for tweet in list_of_tweets:
            processedTweets.append((self._processTweet(tweet["Text"])))
        return processedTweets

    def _processTweet(self, tweet):
        tweet = tweet.lower() # convert text to lower-case
        tweet = re.sub('((www\.[^\s]+)|(https?://[^\s]+))', 'URL', tweet) # remove URLs
        tweet = re.sub('@[^\s]+', 'AT_USER', tweet) # remove usernames
        tweet = re.sub(r'#([^\s]+)', r'\1', tweet) # remove the # in #hashtag
        tweet = word_tokenize(tweet) # remove repeated characters (helloooooooo into hello)
        return [word for word in tweet if word not in self._stopwords]

And when I want to use it:

preprocessedTestSet = tweetProcessor.processTweets(tweet)

I received this output

TypeError: string indices must be integers

What is wrong? How can I fix it?

Answer 1

Assuming tweet is a string. You should pass it as is. You've used tweet["Text"] which is an illegal operation on a string since the index must be an integer.

def processTweets(self, list_of_tweets):
    processedTweets=[]
    for tweet in list_of_tweets:
        processedTweets.append(self._processTweet(tweet))
    return processedTweets

Or more Pythonic:

def processTweets(self, list_of_tweets):
    return [self._processTweet(tweet) for tweet in list_of_tweets]

Notes:

You probably forgot to use raw strings ( r"" ) in some of your regex.