Java Servlets - block all threads using common timer

Question

On Tomcat 6, I have a servlet running which accepts requests and passes these onto an external system.

---

There is a throttling limitation on the external system - if the number of requests exceed a certain number per second, then the external system responds with a Http 503. No further requests may hit the external system for at least 2 seconds or else the external system will restart its throttling timer. Initially, I detected the 503 HttpResponse and did a Thread.sleep(2000) but that is wrong as it doesn't prevent the servlet servicing other requests using other threads - once a 503 response is detected, I need to block all threads for at least the 2 seconds.

---

Ideally, I would prefer the blocked threads not to wake up all at the same time but say a 100ms apart so that requests would be handled in order. I've looked at the Condition and ReentrantLock but unsure if these are appropriate.

Answer 1

Just create a global (static) date variable in the servlet. When you get a 503, change this variable from null to the local time. The servlet should always check this variable before contacting the external system. If the variable is null, or more than 2 seconds have passed, then you can proceed. Otherwise block the thread (or throw an exception).

Answer 2

Looks like calling Amazon services to me, and it can be managed so easy.

You need a central and managed module for doing it, and it comes like a single module.

The important thing is you should not reach the throttling limitation at all, and if you get too much requests which would reach this value, so you should respond to your client check the result later(as async work).

If the request is kinda important business(such as capturing a payment), so you have to implement a failover with the module too, simply by persisting the request data into the database, so if there is any fail, you will have the data from the database.

If you are familiar with MQ arch, so it would be the best solution where they are designed for this kind of stuffs, but you like to have your own, you may accept and process all requests to call teh external system by the module manage.

first you may have a entity class which carries the request info like

class entity{public String id,srv,blah_blah;}

Second, a stand-alone module for accepting and processing the requests, which would be the context for the requests too. like following

class business{private business(){}// fan of OOP? K, go for singleton
private static final ArrayList<entity> ctx=new ArrayList<entity>();
static public void accept_request(entity e){_persist(e);ctx.add(e);}
static private void _persist(entity e){/*persist it to the db*/}
static private void _done(entity e){_remove(e);/*informing 3rd. parties if any*/}
static private void _remove(entity e){/*remove it from the db, it's done*/}
final private static int do_work(e){/*do the real business*/return 0;}//0 as success, 1, fail, 2....
}

But it's not completed yet, now you need a way to call the do_work() guy, so I suggest a background thread(would be daemon too!)

So clients just push the requests to this context-like class, and here we need the thread, like following

class business{...
static public void accept_request(entity e){_persist(e);ctx.add(e);synchronized(ctx){ctx.notify();}}
...
private static final Runnable r=new Runnable(){public void run(){try{
  while(!Thread.currentThread().interrupt()){
    if(ctx.size()==0){synchronized(ctx){if(ctx.size()==0){ctx.wait();}}}
    while(ctx.size()>0){entity e=ctx.get(0);ctx.remove(0);
     if(do_work(e)==0){_done(e);}else{ctx.add(e);/*give him another chance maybe!*/}end-else
    Thread.Sleep(100/*appreciate sleep time*/);}//end-loop

  }
}catch(Throwable wt){/*catch signals, maybe death thread*/}}};
static private Thread t;
void static public start_module(){t=new Thread(r);t.start();}
void static public stop_module(){t.interrupt();t.stop();}
...}

Tip: try not start the thread(calling start_module() ) out of container initiation process, or you will have memory leak! best solution would call the thread by init() method of servlet(s) would call this module(once), and of course halting the the thread by application halt ( destroy() )