Java 8 equivalent of (RxJava) Observable#onComplete()

Question

I'm getting to know Java 8 Stream API and I am unsure how to signal to a consumer of a stream that the stream is completed.

In my case the results of the stream-pipeline will be written in batches to a database or published on a messaging service. In such cases the stream-pipeline should invoke a method to "flush" and "close" the endpoints once the stream is closed.

I had a bit of exposure to the Observable pattern as implemented in RxJava and remember the Observer#onComplete method is used there for this purpose.

On the other hand Java 8 Consumer only exposes an accept method but no way to "close" it. Digging in the library I found a sub-interface of Consumer called Sink which offers an end method, but it's not public. Finally I thought of implementing a Collector which seems to be the most flexible consumer of a stream, but isn't there any simpler option?

Answer 1

The simplest way of doing a final operation is by placing the appropriate statement right after the terminal operation of the stream, for example:

IntStream.range(0, 100).parallel().forEach(System.out::println);
System.out.println("done");

This operation will be performed in the successful case only, where a commit is appropriate. While the Consumer s run concurrently, in unspecified order, it is still guaranteed that all of them have done their work upon normal return.

Defining an operation that is also performed in the exceptional case is not that easy. Have a look at the following example:

try(IntStream is=IntStream.range(0, 100).onClose(()->System.out.println("done"))) {
    is.parallel().forEach(System.out::println);
}

This works like the first one but if you test it with an exceptional case, eg

try(IntStream is=IntStream.range(0, 100).onClose(()->System.out.println("done"))) {
    is.parallel().forEach(x -> {
        System.out.println(x);
        if(Math.random()>0.7) throw new RuntimeException();
    });
}

you might encounter printouts of numbers after done . This applies to all kind of cleanup in the exceptional case. When you catch the exception or process a finally block, there might be still running asynchronous operations. While it is no problem rolling back a transaction in the exceptional case at this point as the data is incomplete anyway, you have to be prepared for still running attempts to write items to the now-rolled-back resource.

Note that Collector -based solutions, which you thought about, can only define a completion action for the successful completion. So these are equivalent to the first example; just placing the completing statement after the terminal operation is the simpler alternative to the Collector .

---

If you want to define operations which implement both, the item processing and the clean up steps, you may create your own interface for it and encapsulate the necessary Stream setup into a helper method. Here is how it might look like:

Operation interface:

interface IoOperation<T> {
    void accept(T item) throws IOException;
    /** Called after successfull completion of <em>all</em> items */
    default void commit() throws IOException {}
    /**
     * Called on failure, for parallel streams it must set the consume()
     * method into a silent state or handle concurrent invocations in
     * some other way
     */
    default void rollback() throws IOException {}
}

Helper method implementation:

public static <T> void processAllAtems(Stream<T> s, IoOperation<? super T> c) 
throws IOException {
    Consumer<IoOperation> rollback=io(IoOperation::rollback);
    AtomicBoolean success=new AtomicBoolean();
    try(Stream<T> s0=s.onClose(() -> { if(!success.get()) rollback.accept(c); })) {
        s0.forEach(io(c));
        c.commit();
        success.set(true);
    }
    catch(UncheckedIOException ex) { throw ex.getCause(); }
}
private static <T> Consumer<T> io(IoOperation<T> c) {
    return item -> {
        try { c.accept(item); }
        catch (IOException ex) { throw new UncheckedIOException(ex); }
    };
}

Using it without error handling might be as easy as

class PrintNumbers implements IoOperation<Integer> {
    public void accept(Integer i) {
        System.out.println(i);
    }
    @Override
    public void commit() {
        System.out.println("done.");
    }
}
processAllAtems(IntStream.range(0, 100).parallel().boxed(), new PrintNumbers());

Dealing with errors is possible, but as said, you have to handle the concurrency here. The following example does also just print number but use a new output stream that should be closed at the end, therefore the concurrent accept calls have to deal with concurrently closed streams in the exceptional case.

class WriteNumbers implements IoOperation<Integer> {
    private Writer target;
    WriteNumbers(Writer writer) {
        target=writer;
    }
    public void accept(Integer i) throws IOException {
        try {
            final Writer writer = target;
            if(writer!=null) writer.append(i+"\n");
            //if(Math.random()>0.9) throw new IOException("test trigger");
        } catch (IOException ex) {
            if(target!=null) throw ex;
        }
    }
    @Override
    public void commit() throws IOException {
        target.append("done.\n").close();
    }
    @Override
    public void rollback() throws IOException {
        System.err.print("rollback");
        Writer writer = target;
        target=null;
        writer.close();
    }
}
FileOutputStream fos = new FileOutputStream(FileDescriptor.out);
FileChannel fch = fos.getChannel();
Writer closableStdIO=new OutputStreamWriter(fos);
try {
    processAllAtems(IntStream.range(0, 100).parallel().boxed(),
                    new WriteNumbers(closableStdIO));
} finally {
    if(fch.isOpen()) throw new AssertionError();
}

Answer 2

Terminal operations on Java 8 streams (like collect() , forEach() etc) will always complete the stream.

If you have something that is processing objects from the Stream you know when the stream ends when the Collector returns.

If you just have to close your processor, you can wrap it in a try-with-resource and perform the terminal operatoni inside the try block

try(BatchWriter writer = new ....){

       MyStream.forEach( o-> writer.write(o));

 }//autoclose writer

Answer 3

You can use Stream#onClose(Runnable) to specify a callback to invoke when the stream is closed. Streams are pull-based (contrary to push-based rx.Observable), so the hook is associated with stream, not it's consumers

Answer 4

This is a bit of a hack but works well. Create a stream concatenation of the original + a unique object.

Using peek(), see if the new object is encountered, and call the onFinish action.

Return the stream with filter, so that the unique object won't be returned.

This preserves the onClose event of the original stream.

public static <T> Stream<T> onFinish(Stream<T> stream, Runnable action) {
    final Object end = new Object(); // unique object

    Stream<Object> withEnd = Stream.concat(stream.sequential(), Stream.of(end));
    Stream<Object> withEndAction = withEnd.peek(item -> {
        if (item == end) {
            action.run();
        }
    });
    Stream<Object> withoutEnd = withEndAction.filter(item -> item != end);
    return (Stream<T>) withoutEnd;
}

Another option is to wrap the original spliterator, and when it returns false, call the action.

public static <T> Stream<T> onFinishWithSpliterator(Stream<T> source, Runnable onFinishAction) {
    Spliterator<T> spliterator = source.spliterator();

    Spliterator<T> result = new Spliterators.AbstractSpliterator<T>(source.estimateSize(), source.characteristics()) {
        @Override
        public boolean tryAdvance(Consumer<? super T> action) {
            boolean didAdvance = source.tryAdvance(action);
            if (!didAdvance) {
                onFinishAction.run();
            }
            return didAdvance;
        }
    };

    // wrap the the new spliterator with a stream and keep the onClose event
    return StreamSupport.stream(result, false).onClose(source::close);

}