Twitter API Rate Limit for Multiple Users

Question

I am writing a PHP-based application using the Twitter API. Up until now I've been using the REST API via a GET request on a PHP page. However, as my app scales, I can easily see it going over the 150 requests-per-hour limit. Here's why:

I have categories of topics, each which periodically poll the Twitter API for tweets around a topic. For example, I have: mysite.com/cars, mysite.com/trucks, etc. A user can go to either page. When he is on the page, live, refreshing updates are pulled from Twitter by making an AJAX call to a PHP page I've set up. The PHP page determines which category the user is coming from (cars, trucks), polls Twitter for search results, then returns the JSON to the category page. This sounds confusing, but there are a number of unrelated reasons I need to have the intermediate PHP page.

The problem is that since the PHP page is making the requests, it will eat up the rate limit very quickly (imagine if there were 20 categories instead of just cars and trucks). I can't make a single call with multiple parameters because it would combine multiple categories of tweets and I'd have no way to separate them. I can cache results, but if I did, the more categories I add, the longer each would have to go between API calls.

So how can I approach this problem? I looked at the streaming API, but it's only for oAuth'd users and I don't want my users to have to log in to anything. Can I use the stream on the PHP page and then just make continuous requests each time the category page polls the PHP page? Thanks for any help!

Answer 1

a) You don't have to use your websites user's oAuth credentials in streaming API - just your's: get them somewhere in dev.twitter.com and hardcode them. Your users won't know there is any oAuth going on backstage.

b) Don't use anonymous requests (150 per IP per hour) use oAuth requests (350 per oAuth per hour). You don't have to ask your users to sing in - just sign in few (1 is sufficient for start) your private twitter accounts. If you don't like creating twitter login functionality, you can get credentials for your twitter account to your twitter application in dev.twitter.com .

c) As @Cheeso mentioned - cache! Don't let every pageload make twitter request.