简体   繁体   中英

Tweepy use multiple API keys with cursor to search Twitter

I've been using the example in this post

to create a system that searches and gets a large number of Tweets in a short time period. However, each time I switch to a new API key (make a new cursor) the search starts all over from the beginning and gets me repeated Tweets. How do I get each cursor to start where the other left off? What am I missing? Here's the code I am using:

currentAPI = 0

a = 0
currentCursor = tweepy.Cursor(apis[currentAPI].search, q = '%40deltaKshatriya')
c = currentCursor.items()
mentions = []
onlyMentions = []
while True:
    try:
        tweet = c.next()
        if a > 100000:
            break
        else:
            onlyMentions.append(tweet.text)
            for t in tTweets:
                if tweet.in_reply_to_status_id == t.id:
                    print str(a) + tweet.text
                    mentions.append(tweet.text)
        a = a + 1
    except tweepy.TweepError:
        print "Rate limit hit"
        if (currentAPI < 9):
            print "Switching to next sat in constellation"
            currentAPI =  currentAPI + 1
            #currentCursor = c.iterator.next_cursor
            currentCursor = tweepy.Cursor(apis[currentAPI].search, q = '%40deltaKshatriya', cursor = currentCursor)
            c = currentCursor.items()
        else:
            print "All sats maxed out, waiting and will try again"
            currentAPI = 0
            currentCursor = tweepy.Cursor(apis[currentAPI].search, q = '%40deltaKshatriya', cursor = currentCursor)
            c = currentCursor.items()
            time.sleep(60 * 15)
        continue
    except StopIteration:
        break

I found a workaround that I think works, although I still encounter some issues. The idea is to add into

currentCursor = tweepy.Cursor(apis[currentAPI].search, q = '%40deltaKshatriya', cursor = currentCursor, max_id = max_id)

Where max_id is the id of the last tweet fetched before the rate limit was hit. The only issue I've encountered is with StopIteration being raised really early (before I get the full 100,000 Tweets) but that I think is a different SO question.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM