使用tweepy进行文本挖掘

Question

I have collected the tweets using tweepy api and i have tokenized them and removed the stopwords but when i load them using json it throws the following error

"File "C:\Python27\Projects\kik.py", line 26, in <module>
    tweet = json.loads(tokens)
  File "C:\Python27\lib\json\__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "C:\Python27\lib\json\decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer" 

Please help me out.

tweets_data_path = 'c:\\Python27\\Projects\\newstweets.txt'
stopset = set(stopwords.words('english'))

tweets_data = []
tweets_file = open(tweets_data_path, "r")
text = tweets_file.read()
tokens=word_tokenize(str(text))
tokens = [w for w in tokens if not w in stopset]
tweet = json.loads(tokens)
tweets_data.append(tweet)

Answer 1

json.loads expects a string, you are trying to load a list.

Instead of:

tokens = [w for w in tokens if not w in stopset]

Try:

tokens = str([w for w in tokens if not w in stopset])