使用 tweepy 获取特定用户的所有推文

Question

I am trying to get all 50k tweets from @realDonaldTrump. I know there is limit for twitter api requests, so I am using max_id=oldest. But I only get 995 tweets.

import tweepy as tweepy

consumerKey = "xxx"
consumerSecret = "xxx"
accessToken = "xxx"
accessTokenSecret = "xxx"

auth = tweepy.OAuthHandler(consumerKey, consumerSecret)
auth.set_access_token(accessToken, accessTokenSecret)

api = tweepy.API(auth, wait_on_rate_limit=True)
alltweets = []

username="@realDonaldTrump"

new_tweets = api.user_timeline(username, tweet_mode = 'extended', count=200)
alltweets.extend(new_tweets)
oldest = alltweets[-1].id - 1

while len(new_tweets) > 0:
    print(f"getting tweets before {oldest}")

    new_tweets = api.user_timeline(username, max_id=oldest,tweet_mode = 'extended', count=200)
    alltweets.extend(new_tweets)
    oldest = alltweets[-1].id - 1

    print(f"...{len(alltweets)} tweets downloaded so far")

outtweets = [[tweet.id_str, tweet.created_at, tweet.full_text] for tweet in alltweets]

Answer 1

Free the free dev account you wont get more than the last 3200 tweets.

I suggest to use cursor and pages.

..
c = tw.Cursor(api.user_timeline, id=userid, tweet_mode="extended", wait_on_rate_limit=True,count=200).pages()
while True:
    try:
        page = c.next()
        tweets.extend(page)
..
    except tw.TweepError:
        print(e)
        time.sleep(60)
        continue
    except StopIteration:
        break