[英]Tweepy: get old tweets now possible with Twitter search api?
Accoring to http://www.theverge.com/2014/11/18/7242477/twitter-search-now-lets-you-find-any-tweet-ever-sent Twitter search now lets you find any tweet ever sent. 根据http://www.theverge.com/2014/11/18/7242477/twitter-search-now-lets-you-find-any-tweet-ever-sent的 Twitter搜索,现在可以查找曾经发送过的任何推文。
But when i am trying to get tweets from 2014 to 2015 using tweepy it gets only recent: 但是,当我尝试使用tweepy获取2014年至2015年的推文时,它仅是最近的:
query = 'Nivea'
max_tweets = 1000
searched_tweets = [json.loads(status.json) for status in tweepy.Cursor(api.search,
q=query,
count=100,
#since_id="24012619984051000",
since="2014-02-01",
until="2015-02-01",
result_type="mixed",
lang="en"
).items(max_tweets)]
I tried since="2014-02-01", and since_id but no matter. 我尝试了since =“ 2014-02-01”和since_id,但没关系。
Unfortunately, you cannot access past data from Twitter. 不幸的是,您无法从Twitter访问过去的数据。 Is not a problem of what library you're using: Tweepy, Twitter4J, whatever, is just that Twitter won't provide any data that is older than more or less 2 weeks.
所使用的库不是问题:Tweepy,Twitter4J,无论如何,仅仅是Twitter不会提供早于或少于2周的任何数据。
To get historical data you'll need access to firehose, directly through Twitter or third-party resellers like GNIP. 要获取历史数据,您需要直接通过Twitter或GNIP之类的第三方经销商访问firehose。
I use my own piece of code which uses a HttpURLConnection
and a twitter search url. 我使用自己的一段代码,该代码使用
HttpURLConnection
和Twitter搜索URL。 I then use a regular expression to pull out the last 20 matching tweets... Luckily as I'm deleting the tweets I can simply search again until I can't find anymore tweets. 然后,我使用正则表达式提取出最后20条匹配的推文...幸运的是,当我删除这些推文时,我可以简单地再次搜索,直到找不到任何推文为止。 I'm including the code although it's in Java but the same would apply for any language.
我包含了代码,尽管它是用Java编写的,但同样适用于任何语言。 First I use a class to actually search for tweets and record their details:
首先,我使用一个类来实际搜索推文并记录其详细信息:
public class ReadSearch{
private String startURL = "https://twitter.com/search?f=realtime&q=from%3A";
private String middleURL = "%20%40";
private String endURL = "&src=typd";
public ArrayList<Tweet> getTweets(String user, String troll) {
ArrayList<Tweet> tweets = new ArrayList<Tweet>();
String expr = "small.class=\"time\".*?href=\"/"
+ "([^/]+)"
+ ".*?status/"
+ "([^\"]+)"
+ ".*?title=\""
+ "([^\"]+)";
Pattern patt = Pattern.compile(expr, Pattern.DOTALL | Pattern.UNIX_LINES);
try {
Matcher m = patt.matcher(getData(startURL+user+middleURL+troll+endURL));
while (m.find()) {
if(user.equals(m.group(1).trim())){
Tweet tw = new Tweet();
tw.setUser(m.group(1).trim());
tw.setTweetid(Long.parseLong(m.group(2).trim()));
tw.setDate(m.group(3).trim());
tweets.add(tw);
}
}
} catch (Exception e) {
e.printStackTrace();
System.out.println("Exception " + e);
}
return tweets;
}
private StringBuilder getData(String dataurl) throws MalformedURLException, IOException{
URL url = new URL(dataurl);
HttpURLConnection httpcon = (HttpURLConnection) url.openConnection();
httpcon.addRequestProperty("User-Agent", "Mozilla/4.76");
StringBuilder sb = new StringBuilder(16384);
BufferedReader br = new BufferedReader(new InputStreamReader(httpcon.getInputStream(), "ISO-8859-1"));
String line;
while ((line = br.readLine()) != null){
sb.append(line);
sb.append('\n');
}
httpcon.disconnect();
br.close();
return sb;
}
public static void main(String [] args){
//testing
ReadSearch rs = new ReadSearch();
ArrayList<Tweet> tweets = rs.getTweets("Tony_Kennah", "PickLuckier");
for(Tweet t : tweets){
System.out.println("TWEET: " + t.toString());
}
}
}
We then need the Tweet class itself so we can group Tweets up and do things with them, it's just a bean like this: 然后,我们需要Tweet类本身,以便我们可以将Tweets分组并与它们一起执行操作,它只是一个像这样的bean:
public class Tweet{
private String user;
private long tweetid;
private String date;
public String getUser(){
return user;
}
public void setUser(String user){
this.user = user;
}
public long getTweetid(){
return tweetid;
}
public void setTweetid(long tweetid){
this.tweetid = tweetid;
}
public String getDate(){
return date;
}
public void setDate(String date){
this.date = date;
}
public String toString(){
return this.tweetid + " " + this.user + " " + this.date;
}
}
... and so that was all just standard java. ...而这仅仅是标准的Java。 To make use of the above code I use the Twitter4J API and do this:
要使用上面的代码,我使用Twitter4J API并执行以下操作:
public class DeleteTweets
{
public static void main(String args[]) throws Exception
{
Twitter twitter = TwitterFactory.getSingleton();
ArrayList<Tweet> tweets = new ArrayList<Tweet>();
String [] people = { "PickLuckier" };
for(String s : people){
do{
ReadSearch rs = new ReadSearch();
tweets = rs.getTweets(twitter.getScreenName(), s);
for(Tweet tw : tweets){
twitter.destroyStatus(tw.getTweetid());
}
} while(tweets.size()!=0);
}
}
}
That's it. 而已。 I don't use comments but I hope it's easy to see what's going on and that this helps you out.
我不使用评论,但我希望很容易看到正在发生的事情,这对您有所帮助。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.