Wait till a Blocking Queue is full

Question

Im looking for a way to synchronize multiple asynchronous operations. I'd like to use a BlockingQueue with a size equal to my operations but who can i wait till the Queue is full?

Im looking for something like a reversed Blocking Queue.

I need to gather the Results of each Thread at the End. The AsyncHandler is fixed, its already a ThreadExecutor underlying, i cannot start new Threads.

//3 Times
makeAsync(new AsyncHandler() {
   onSuccess() {
     ..
     queue.put(result)
   }

   onFailure() {
     ..
   }
});

//Blocking till the Queue is full
List<Results> = queue.takeAll()

Bonus Question: I need a way to end the wait when one of my Requests fails

Answer 1

我从来不需要做这种事情，但是您可能会从各个线程使用CountDownLatch或CyclicBarrier感到有些幸运。

Answer 2

If you're using Java 8, do the following:

• With each call to makeAsync , create a CompletableFuture<Result> instance and make it available to the AsyncHandler , and have the caller keep a reference too, say in a list.
• When an async task completes normally, have it call complete(result) on its CompletableFuture instance.
• When an async task completes with an error, have it call completeExceptionally(exception) on its CompletableFuture instance.
• After initiating all the asynchronous tasks, have the caller call CompletableFuture.allOf(cfArray).join() . Unfortunately this takes an array, not a list, so you have to convert. The join() call will throw an exception if any one of the tasks completed with an error. Otherwise, you can collect the results from the individual CompletableFuture instances by calling their get() methods.

If you don't have Java 8, you'll have to sort of roll your own mechanism. Initialize a CountDownLatch to the number of async tasks you're going to fire off. Have each async task store its result (or an exception, or some other means of indicating failure) into a thread-safe data structure and then decrement ('countDown`) the latch. Have the caller wait for the latch to reach zero and then collect the results and errors. This isn't terribly difficult, but you have to determine a means for storing valid results as well as recording whether an error occurred, and also maintain a count manually.

Answer 3

What you describe with

//Blocking till the Queue is full
List<Results> results = queue.takeAll();

does not differ semantically from “take as much items as the queue's capacity”. If you know the capacity you can achieve this by:

// preferably a constant which you also use to construct the bounded queue
int capacity;

…

List<Results> results = new ArrayList<>(capacity);
queue.drainTo(results, capacity);
while(result.size()<capacity)
    queue.drainTo(results, capacity-result.size());

This will block until it has received as much items as the capacity which is, as said, the same as waiting for the queue to become full (has a size equal to its capacity) and than take all items. The only difference is that the event of the queue becoming full is not guaranteed to happen, eg if you intend your async operations to offer items until the queue is full, it does not work this way.

If you don't know the capacity, you are out of luck. There is not even a guaranty that an arbitrary BlockingQueue is bounded, read, it might have an unlimited capacity.

On the other hand, if the asynchronous operations are able to detect when they have finished, they could simply collect the items in a list locally and put the entire list into a BlockingQueue<List<Results>> as a single item once they are done. Then your code waiting for it needs only a single take to get the entire list.

Answer 4

If you can modify methodAsync(), then it's as simple as to use a CountDownLatch after each time you put some elements in the queue and have the main thread wait for such a CountDownLatch.

If unfortunately you cannot modify methodAsync(), then simply wrap the queue and give it a count down latch, and then override the add() method to count down this latch. The main method just wait it to be done.

Having said the above, your program structure smells not well organized.