How to implement rate limiting based on a client token in Spring?

Question

I am developing a simple REST API using Spring 3 + Spring MVC. Authentication will be done through OAuth 2.0 or basic auth with a client token using Spring Security. This is still under debate. All connections will be forced through an SSL connection.

I have been looking for information on how to implement rate limiting, but it does not seem like there is a lot of information out there. The implementation needs to be distributed, in that it works across multiple web servers.

Eg if there are three api servers A, B, C and clients are limited to 5 requests a second, then a client that makes 6 requests like so will find the request to C rejected with an error.

A recieves 3 requests   \
B receives 2 requests    | Executed in order, all requests from one client.
C receives 1 request    /

It needs to work based on a token included in the request , as one client may be making requests on behalf of many users, and each user should be rate limited rather than the server IP address.

The set up will be multiple (2-5) web servers behind an HAProxy load balancer. There is a Cassandra backed, and memcached is used. The web servers will be running on Jetty.

One potential solution might be to write a custom Spring Security filter that extracts the token and checks how many requests have been made with it in the last X seconds. This would allow us to do some things like different rate limits for different clients.

Any suggestions on how it can be done? Is there an existing solution or will I have to write my own solution? I haven't done a lot of web site infrastructure before.

Answer 1

It needs to work based on a token included in the request, as one client may be making requests on behalf of many users, and each user should be rate limited rather than the server IP address.

The set up will be multiple (2-5) web servers behind an HAProxy load balancer. There is a Cassandra backed, and memcached is used. The web servers will be running on Jetty.

I think the project is request/response http(s) protocol. And you use HAProxy as fronted. Maybe the HAProxy can load balancing with token , you can check from here .

Then the same token requests will reach same webserver, and webserver can just use memory cache to implement rate limiter.

Answer 2

I would avoid modifying application level code to meet this requirement if at all possible.

I had a look through the HAProxy LB documentation nothing too obvious there, but the requirement may warrant a full investigation of ACLs.

Putting HAProxy to one side, a possible architecture is to put an Apache WebServer out front and use an Apache plugin to do the rate limiting. Over-the-limit requests are refused out front and the application servers in the tier behind Apache are then separated from rate limit concerns making them simpler. You could also consider serving static content from the Web Server.

See the answer to this question How can I implement rate limiting with Apache? (requests per second)

I hope this helps. Rob

Answer 3

You could put rate limits at various points in the flow (generally the higher up the better) and the general approach you have makes a lot of sense. One option for the implementation is to use 3scale to do it ( http://www.3scale.net ) - it does rate limits, analytics, key managed etc. and works either with a code plugin (the Java plugin is here: https://github.com/3scale/3scale_ws_api_for_java ) which pushes or by putting something like Varnish (http://www.varnish-cache.org) in the pipeline and having that apply rate limits.

Answer 4

I was also thinking of the similar solutions a couple of day's ago. Basically, I prefer the "central-controlled" solution to save the state of the client request in the distributed environment.

In my application, I use a "session_id" to identify the request client. Then create a servlet filter or spring HandlerInterceptorAdapter to filter the request, then check the "session_id" with the central-controlled data repository, which could be memcached, redis, cassandra or zookeeper.

Answer 5

We use redis as leaky bucket backend

Add a controller as entrance

google cache that token as key with expired time

then filter every request

Answer 6

It is best if you implement ratelimit using REDIS . For more info please look this Rate limiting js Example.