Intercepting outgoing servlet http requests

Question

I am working on a school project in which I query and receive some fairly large XML documents from a central server. This was fine in the beginning, as I was rarely making these requests (HTTP GET), but as the project progressed I came up with more things to do with this data, and now I have servlets requesting 3 or 4 XML documents, each in it's own separate GET request, which is causing upwards of 25 seconds page generation times.

It's not possible to change the way the data is served, neither the way in which it's requested as I have a fairly large code base, and it's not as decoupled as it perhaps should have been.

Is there a smart way to listen in on when my servlets execute these GET requests, intercept them, and perhaps supply them with a local, cached version instead? The data is not THAT volatile, so 'live' data is not needed.

So far, I have not been able to find information on listening on OUTgoing requests made by Tomcat...

Answer 1

I think a lot will depend upon your cache hit ratio. If the same 3-4 documents (or some small group of documents) are being requested on a regular basis, a local caching proxy server (like Squid) might be a possibility. Java can be configured to use a proxy server for HTTP requests.

Answer 2

You can probably implement this using HttpFilter. It can be used as a cache. If requested document is already in cache just return it; if not forward the HTTP request to your servlet.

Answer 3

I ended up using a ContextListener to load most of the data at start-up in addition to an 'expiry date' into the servlet context attributes. It makes for some slow start-ups (9 GetRequests to the central server!) but drastically reduces our page load times.