Tweepy错误，响应状态码为429

Question

I am using tweepy API to pull tweets and display it on the webpage. Although I auto-refresh the code to extract tweets every 1 hour, I end up with the following TweepError after about 24 hours of the code continuously running. Error:

tweepy.error.TweepError: Twitter error response: status code = 429

I understand that this code implies that there are too many requests to the API. Is there any way to handle this error? I have even tried increasing the refresh rate interval to 2 hours, and still doesn't seem to work.

Below is the code to extract tweets and store in CSV:

def extract_tweet(self):
   consumer_key = 'XXX'
   consumer_secret = 'XXX'
   access_token = 'YYY'
   access_token_secret = 'YYY'

   auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
   auth.set_access_token(access_token, access_token_secret)

   api = tweepy.API(auth)

   csvFile = open('HELLOTWITTER.csv', 'w',  newline='', encoding='utf-8')
   csvWriter = csv.writer(csvFile)
   for tweet in tweepy.Cursor(api.search,q="#cybersecurity OR #InfoSec", since="2017-10-20",lang="en").items(10):    
       csvWriter.writerow([tweet.text, tweet.created_at])
   csvFile.close()

I have looked at the API documentation and stumbled upon wait_on_rate_limit , but haven't understood if that is the solution to this error. Please help! Thanks.

Answer 1

There are several options to implement retrying decorator. Retry decorator looks if an exception is raised in the method. Following example uses exponential backoff retrying. This will wait 2^x * 1000 milliseconds between each retry, up to 10 seconds, then 10 seconds afterwards, where x is the number of retry attempt.

from retrying import retry

@retry(wait_exponential_multiplier=1000, wait_exponential_max=10000)
def extract_tweet(self):
    consumer_key = 'XXX'
    consumer_secret = 'XXX'
    access_token = 'YYY'
    access_token_secret = 'YYY'

    try:
        auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
        auth.set_access_token(access_token, access_token_secret)

        api = tweepy.API(auth)

        csvFile = open('HELLOTWITTER.csv', 'w',  newline='', encoding='utf-8')
        csvWriter = csv.writer(csvFile)

        for tweet in tweepy.Cursor(api.search,q="#cybersecurity OR #InfoSec", since="2017-10-20",lang="en").items(10):    
          csvWriter.writerow([tweet.text, tweet.created_at])
          csvFile.close()
    except tweepy.error.TweepError:
          raise