如何提取@ASPCA twitter 页面的所有推文和回复？

Question

I am going to extract all tweets and replies. I used the following code but it only extract about 3.2K tweets.

import pandas as pd
import numpy as np
import tweepy 

consumer_key = "*"                
consumer_secret = "*"                   
access_token = "*"
access_token_secret = "*"

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth,wait_on_rate_limit=True)

username = 'ASPCA'

tweets = tweepy.Cursor(api.user_timeline, screen_name=username).items()
tweets_list = [[tweet.created_at, tweet.id, tweet.text, tweet.in_reply_to_status_id_str,         tweet.in_reply_to_screen_name, tweet.in_reply_to_user_id_str, tweet.user.id] for tweet in tweets]

The page has almost 34K tweets.

I appreciate anyone to help

Answer 1

That's correct. The Twitter user_timeline API provides access to 3200 Tweets maximum. This is the documented limit to the API.

There are really two options here:

• ask the account owner to download their Twitter archive and provide you with access to the data (which I assume is not an option in this case)

• use the Twitter premium full-archive search API to search for all of the Tweets sent by the user. Since there are so many Tweets and this account is over ten years old, I would expect this to require a large number of requests, so the free sandbox tier of the API is unfortunately unlikely to work for you in this case.