How to avoid rate-limiting 429 error in Twython

Question

I've created a function designed to run through a column of Twitter handles pandas dataframe, yet it always seems to hit rate-limiting error after just 14 calls.

Here's the code.

def poll_twitter(dfr):
    followers  = twitter.get_followers_ids(screen_name = dfr['handle'])
    time.sleep(5)
    print "looping..."
    return len(followers['ids'])

df[datetime.datetime.today()] = df.apply(poll_twitter, axis=1)    

Here's the error

TwythonRateLimitError: (u'Twitter API returned a 429 (Too Many Requests), Rate limit exceeded'

The list is only 100 handles so I assumed there would be plenty of calls available.

What's the way of fixing it?

Answer 1

Twitter GET followers/ids endpoint in API 1.1 version has 15 requests/per window (15 mins) limit, ie about 60 requests per hour.

Note also, that it also returns up to 5000 ids per request, so you have to issue more requests for highly followed users. For example only Barack Obama followers list will take 40434976/(5000*60*24) = 5.62 days to be loaded.

Answer 2

您可以使用 Twython 的 get_lastfunction_header('x-rate-limit-remaining')。