在Java Web应用程序中循环调用Web服务

Question

We have an external SOAP based web-service which provides information regarding customer's gift card balance when presented with an Id. This Id is stored in our database.

The requirement is to find out the balance for all such customers who has this Id flagged and then send them an email. This logic is supposed to be run as a scheduled job once every alternate day.

When we queried the DB, we found out that there are more than 5000 such customers who have this Id flagged. Unfortunately, the web-service will NOT accept a list of Ids, and can only give information about a single customer in one network call.

Now, our doubt is whether it will be a good idea to loop through 5000 Ids and call the web-service in this loop as many times. As a test run, when we called the web-service for 500 Ids, it completed in 3.7 minutes and 1000 Ids 7.25 minutes. By this measure, we can guesstimate that for 5000 Ids, it should roughly take 40 minutes.

Our web-application is JavaEE 6 stack and DB is Oracle.

Is there a better way to do this ? Any suggestions are welcome. Thanks.

Answer 1

Unfortunately, the web-service will NOT accept a list of Ids, and can only give information about a single customer in one network call.

You should really take contact with the service provider to get a suitable solution.

As workaround, if making multiple concurrent invocations is allowed by the SOAP WS, you could make multiple invocations of the WS by multiple Threads.
To achieve that, create a Runnable or a Callable implementation that performs the invocation to the WS with a specific id.

For example to perform concurrently 10 invocations of the WS, with Callable and ExecutorService, you could do something as :

MyWs myWs = ...; // web service stub
List<Long> ids = ...; // ids to search

List<Callable<Double>> callables = ids.stream()
                                      .map(id -> (Callable<Double>) () -> myWs.getBalance(id))
                                      .collect(Collectors.toList());

ExecutorService executorService = Executors.newFixedThreadPool(10)
List<Future<Double>> balanceFutures = executorService.invokeAll(callables);

Of course adjust the number of invocations according to the CPU of the machine that runs the JVM.

Answer 2

If you could write a deterministic function that takes the input of the customer id and that gives you a number from 0 to 47 representing the number of hours in the 2 day cycle of sending these email alerts, you could shard the email sending and convert it to a job that runs every hour.

I know that is changing the requirements a bit, but there isn't much difference between sending a batch every 2 days and a smaller batch every hour. Each customer who remains on your list would continue to get emails every 2 days.

Another possibility is to send queries to the web service in a multi-threaded manner.
The web service provider should really think about changing their interface.