LongAccumulator does not get a right result

Question

Code first:

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.LongAccumulator;
import java.util.function.LongBinaryOperator;
import java.util.stream.IntStream;

/**
 * Created by tom on 17-4-13.
 */
public class ForSOF {
    public static void main(String[] args){
        LongBinaryOperator op = (v, y) -> (v*2 + y);
        LongAccumulator accumulator = new LongAccumulator(op, 1L);

        ExecutorService executor = Executors.newFixedThreadPool(2);

        IntStream.range(0, 10)
                .forEach(i -> executor.submit(() -> accumulator.accumulate(i)));

        stop(executor);

        // 2539 expected, however result does not always be! I had got 2037 before.
        System.out.println(accumulator.getThenReset());
    }

    /**
     * codes for stop the executor, it's insignificant for my issue.
     */
    public static void stop(ExecutorService executor) {
        try {
            executor.shutdown();
            executor.awaitTermination(60, TimeUnit.SECONDS);
        }
        catch (InterruptedException e) {
            System.err.println("termination interrupted");
        }
        finally {
            if (!executor.isTerminated()) {
                System.err.println("killing non-finished tasks");
            }
            executor.shutdownNow();
        }
    }
}

Those above are whole codes what I crash about testing LongAccumulator class of Java 8. Codes from blog winterbe.com with nice tourist articles about java8.

As API document said, method of those class in package java.util.concurrent.atomic will be atomic and thread-safe. However result was certainly when I run those above. Some time erorr with 2037 or 1244. There must be something wrong.

When I rewrite the first line inside main body to this:

LongBinaryOperator op = (v, y) -> (v + y);

It works in right way.

So, I guess that when JVM get v and double it, other thread had interrupt and get a same value of v, compute op expression and give a result which be rewrite by pre thread soon. That's why expre v+y works.

If anywhere of those codes should be improved? Or a bug of LongAccumulator class? I'm not sure.

---

God! What the logic of those code should be this:

public void test(){
    int v = 1;
    for (int i = 0; i < 10; i++) {
        v = x(v, i);
    }
    System.out.println(v);
}

int x(int v, int y){
    return v*2+y;
}

and output of v is 2037.

so what confuse me must be the disorder of computing stream.

Answer 1

As API document said, method of those class in package java.util.concurrent.atomic will be atomic and thread-safe. However result was certainly when I run those above. Some time error with 2037 or 1244. There must be something wrong.

The LongAccumulator docs provide one important piece of information about this.

The order of accumulation within or across threads is not guaranteed and cannot be depended upon, so this class is only applicable to functions for which the order of accumulation does not matter.

Your problem is that v * 2 + y operation is noncommutative . If you mix the order of the operations, you can get a wide variety of output. The reason why v + y works is that you can apply that in any order and get the same result.

To break it out into serial code, you are effectively doing:

List<Long> list = new ArrayList<>();
for (long i = 0; i < 10; i++) {
    list.add(i);
}
// if you uncomment this, you get varying results
// Collections.shuffle(list);
long total = 1;
for (Long i : list) {
    total = total * 2 + i;
}
System.out.println(total);

The in-order result is 2037 but if you uncomment the Collections.shuffle(...) then you will see that the result varies with a max of 9218 resulting when the numbers are in reverse order.

Answer 2

Update: The problem described below has been accepted as a bug: JDK-8178956

Since this is a question about whether LongAccumulator is buggy, I'd like to point out that it is, or perhaps rather that the documentation might be wrong.

First, though, the documentation clearly states:

The order of accumulation within or across threads is not guaranteed and cannot be depended upon, so this class is only applicable to functions for which the order of accumulation does not matter.

The function used in the question ( (v, y) -> (v*2 + y) ) clearly violates that, since the order of calls to accumulate() matters.

To put it more in math form, if we accumulate two values, a1 and a2 , the documentation clearly states that the result can be either of these:

result = f( f(identity, a1), a2)
result = f( f(identity, a2), a1)

The documentation also states:

The supplied accumulator function should be side-effect-free, since it may be re-applied when attempted updates fail due to contention among threads.

Which simply means that f(identity, a1) may be call twice, but the result of one of the calls will be discarded. Or that both f(identity, a1) and f(identity, a2) are called, but only one of them is used like shown above, and the other is discarded.

But, the documentation then says:

The function is applied with the current value as its first argument, and the given update as the second argument.

Which is what is shown above, however that is wrong , because the result may also be calculated like this, in clear violation of that last statement:

result = f( identity, f(a1, a2) )

where the first argument to the inner call is not the "current value" of the accumulator, and the second argument to the outer call is not the "given update" value.

To see this, I modified the question code to this:

LongBinaryOperator op = (v, y) -> {
    try { Thread.sleep(200); } catch (InterruptedException ignored) {}
    long result = (v + y);
    System.out.printf("f(%d, %d) = %d%n", v, y, result);
    return result;
};
LongAccumulator accumulator = new LongAccumulator(op, 1L);

ExecutorService executor = Executors.newFixedThreadPool(2);

Arrays.asList(2, 3, 5, 11, 17, 23, 29, 37, 41, 47).stream()
        .forEach(i -> executor.submit(() -> accumulator.accumulate(i)));

executor.shutdown();
executor.awaitTermination(60, TimeUnit.SECONDS);

System.out.println(accumulator.getThenReset());

Running it produced this result (result varies on every execution):

f(1, 3) = 4
f(1, 2) = 3
f(2, 11) = 13
f(4, 5) = 9
f(13, 17) = 30
f(23, 29) = 52
f(30, 37) = 67
f(52, 41) = 93
f(67, 47) = 114
f(9, 93) = 102
f(102, 114) = 216
216

To show how the function is called to calculate the result, I'll annotate it by applying names to the values, ie id for initial identity value of 1 , a0 to a9 for the accumulate (input) values, and r1 to r11 for the function results. Note that r2 is discarded, which is why there are 11 calls with only 10 values.

f(  1 id ,   3 a1) =   4 R1
f(  1 id ,   2 a0) =   3 R2  // result discarded
f(  2 a0 ,  11 a3) =  13 R3  // both arguments are input values
f(  4 R1 ,   5 a2) =   9 R4
f( 13 R3 ,  17 a4) =  30 R5
f( 23 a5 ,  29 a6) =  52 R6  // both arguments are input values
f( 30 R5 ,  37 a7) =  67 R7
f( 52 R6 ,  41 a8) =  93 R8
f( 67 R7 ,  47 a9) = 114 R9
f(  9 R4 ,  93 R8) = 102 R10  // both arguments are result value
f(102 R10, 114 R9) = 216 R11  // both arguments are result value

Answer 3

You have to change your code as shown below:

public static void main(String[] args) {
    LongBinaryOperator op = (v, y) -> (v + 2*y);
    LongAccumulator accumulator = new LongAccumulator(op, 1L);
    ExecutorService executor = Executors.newFixedThreadPool(4);
    IntStream.range(0, 100)
            .mapToObj(val -> executor.submit(() -> accumulator.accumulate(val)))
            .forEach(future -> { while (!future.isDone()); });
    executor.shutdown();
    System.out.println(accumulator.get());
}

First it makes op execution order independent, because v represents the sum of already calculated values and y the value of your IntStream for which to calculate the new value 2*y :

LongBinaryOperator op = (v, y) -> (v + 2*y);

Second you have to wait for each submitted task to finish, which can be detected by inspecting the returned Future :

forEach(future -> { while (!future.isDone()); });

Now it should work stable.