如何从Twitter Rest API存储JSON对象

Question

Hey i wrote a python script to store some data from the Twitter Rest API to my mongodb. It worked for the streaming API but now i got an Error for the Rest API. Can someone help me?

Here is my code and the Error:

Code:

 searchQuery = '#python'  # this is what we're searching for
    maxTweets = 10000000 # Some arbitrary large number
    tweetsPerQry = 100  # this is the max the API permits

    sinceId = None
    max_id = -1L

client = MongoClient(MONGO_HOST)
db = client.alphaQuestDB

tweetCount = 0
print("Downloading max {0} tweets".format(maxTweets))
while tweetCount < maxTweets:
    try:
        if (max_id <= 0):
            if (not sinceId):
                new_tweets = api.search(q=searchQuery, count=tweetsPerQry)
            else:
                new_tweets = api.search(q=searchQuery, count=tweetsPerQry,
                                        since_id=sinceId)
        else:
            if (not sinceId):
                new_tweets = api.search(q=searchQuery, count=tweetsPerQry,
                                        max_id=str(max_id - 1))
            else:
                new_tweets = api.search(q=searchQuery, count=tweetsPerQry,
                                        max_id=str(max_id - 1),
                                        since_id=sinceId)
        if not new_tweets:
            print("No more tweets found")
            break
        #datajson = json.loads(new_tweets)
        #created_at = datajson['created_at']
        for tweet in new_tweets:
            print(tweet)
            print(type(tweet))
            datajson = json.loads(tweet)
            db.alphaCollection.insert(datajson)
   tweetCount += len(new_tweets)
        print("Downloaded {0} tweets".format(tweetCount))
        max_id = new_tweets[-1].id
    except tweepy.TweepError as e:
        # Just exit if any error
        print("some error : " + str(e))
        break



print("Downloaded {0} tweets, Saved to {1}".format(tweetCount, 'MongoDB'))

And here is the Error:

<class 'tweepy.models.Status'>
Traceback (most recent call last):
  File "/home/krodi/eclipse-workspace/alphaQuestTechTrends/Twitter_REST_API.py", line 59, in <module>
    datajson = json.loads(tweet)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

Answer 1

Are there spaces in the keywords you are tracking? If so, suggest trying to remove any keywords with a space and place keywords in single quotes. This is a closed issue with the tweepy repo (see #748): https://github.com/tweepy/tweepy/issues/748