How do I solve this Python KeyError?

Question

So, I am trying to build a twitter sentiment analysis model using naive bayes. I'm getting a problem while I'm trying to preprocess my data. Here is my code:

This is my data loading code

df = pd.read_csv('full-corpus.csv',
encoding='latin1',
names=['topic', 'sentiment', 'TweetId', 'TweetDate','TweetText'])
trainingData = df.to_dict(orient='records')

This is my code for data preprocessing:

import re
import nltk
nltk.download('stopwords')
nltk.download('punkt')
from nltk.tokenize import word_tokenize
from string import punctuation 
from nltk.corpus import stopwords 

class PreProcessTweets:
    def __init__(self):
        self._stopwords = set(stopwords.words('english') + list(punctuation) + ['AT_USER','URL'])
        
    def processTweets(self, list_of_tweets):
        processedTweets=[]
        for tweet in list_of_tweets:
            
            processedTweets.append((self._processTweet(tweet['TweetText']),tweet['sentiment']))
        return processedTweets
    
    def _processTweet(self, tweet):
        tweet = tweet.lower() # convert text to lower-case
        tweet = re.sub('((www\.[^\s]+)|(https?://[^\s]+))', 'URL', tweet) # remove URLs
        tweet = re.sub('@[^\s]+', 'AT_USER', tweet) # remove usernames
        tweet = re.sub(r'#([^\s]+)', r'\1', tweet) # remove the # in #hashtag
        tweet = word_tokenize(tweet) # remove repeated characters (helloooooooo into hello)
        return [word for word in tweet if word not in self._stopwords]

tweetProcessor = PreProcessTweets()
preprocessedTrainingSet = tweetProcessor.processTweets(trainingData)
preprocessedTestSet = tweetProcessor.processTweets(testDataSet)

I'm getting this keyerror

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-96-acc96635cf33> in <module>()
      1 tweetProcessor = PreProcessTweets()
      2 preprocessedTrainingSet = tweetProcessor.processTweets(trainingData)
----> 3 preprocessedTestSet = tweetProcessor.processTweets(testDataSet)

<ipython-input-95-05e6d1942355> in processTweets(self, list_of_tweets)
     15         for tweet in list_of_tweets:
     16 
---> 17             processedTweets.append((self._processTweet(tweet['TweetText']),tweet['sentiment']))
     18         return processedTweets
     19 

KeyError: 'TweetText'

Please help to solve this. Thanks

Answer 1

It seems like you pass through a list, but you want to use a dict. For it to work you have to use:

for key, value in dict_foo.items()

If you go through a list while it is a dict you will just have access to the key.