堆栈内存溢出
  繁体   English   中英

Java并行流内部

[英]Java parallel stream internals

 原文  2017-11-26 19:40:43  6   2    java/ multithreading/ parallel-processing/ java-8/ java-stream

问题描述

我注意到这取决于doSth（）方法的实现（如果线程休眠一段时间或一段时间），并行执行不同的并行流。

例： 

import java.util.Random;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.stream.IntStream;

import static java.lang.System.out;

public class AtomicInt {

    public static void main(String[] args) throws ExecutionException, InterruptedException {
        out.println("Result: " + count());
    }

    public static int count() throws ExecutionException, InterruptedException {
        ForkJoinPool forkJoinPool = new ForkJoinPool(10);

        AtomicInteger counter = new AtomicInteger(0);

        forkJoinPool.submit(() -> IntStream
                .rangeClosed(1, 20)
                .parallel()
                .map(i -> doSth(counter))
                .forEach(i -> out.println(">>>forEach: " + Thread.currentThread().getName() + " value: " + i))
        ).get();

        return counter.get();
    }

    private static int doSth(AtomicInteger counter) {
        try {
            out.println(">>doSth1: " + Thread.currentThread().getName());
            Thread.sleep(100 + new Random().nextInt(1000));
//            Thread.sleep(1000);
        } catch (InterruptedException e) {
            throw new RuntimeException(e);
        }

        int counterValue = counter.incrementAndGet();
        out.println(">>doSth2: " + Thread.currentThread().getName() + " value: " + counterValue);

        return counterValue;
    }
}

每个号码都按顺序处理： 

>>doSth1: ForkJoinPool-1-worker-9
>>doSth1: ForkJoinPool-1-worker-8
>>doSth1: ForkJoinPool-1-worker-2
>>doSth1: ForkJoinPool-1-worker-1
>>doSth1: ForkJoinPool-1-worker-6
>>doSth1: ForkJoinPool-1-worker-11
>>doSth1: ForkJoinPool-1-worker-15
>>doSth1: ForkJoinPool-1-worker-4
>>doSth1: ForkJoinPool-1-worker-13
>>doSth1: ForkJoinPool-1-worker-10
>>doSth2: ForkJoinPool-1-worker-8 value: 1
>>>forEach: ForkJoinPool-1-worker-8 value: 1
>>doSth1: ForkJoinPool-1-worker-8
>>doSth2: ForkJoinPool-1-worker-15 value: 2
>>>forEach: ForkJoinPool-1-worker-15 value: 2
>>doSth1: ForkJoinPool-1-worker-15
>>doSth2: ForkJoinPool-1-worker-11 value: 3
>>>forEach: ForkJoinPool-1-worker-11 value: 3
>>doSth1: ForkJoinPool-1-worker-11
>>doSth2: ForkJoinPool-1-worker-2 value: 4
>>>forEach: ForkJoinPool-1-worker-2 value: 4
>>doSth1: ForkJoinPool-1-worker-2
>>doSth2: ForkJoinPool-1-worker-9 value: 5
>>>forEach: ForkJoinPool-1-worker-9 value: 5
>>doSth1: ForkJoinPool-1-worker-9
>>doSth2: ForkJoinPool-1-worker-11 value: 6
>>>forEach: ForkJoinPool-1-worker-11 value: 6
>>doSth1: ForkJoinPool-1-worker-11
>>doSth2: ForkJoinPool-1-worker-1 value: 7
>>>forEach: ForkJoinPool-1-worker-1 value: 7
>>doSth1: ForkJoinPool-1-worker-1
>>doSth2: ForkJoinPool-1-worker-15 value: 8
>>>forEach: ForkJoinPool-1-worker-15 value: 8
>>doSth1: ForkJoinPool-1-worker-15
>>doSth2: ForkJoinPool-1-worker-8 value: 9
>>>forEach: ForkJoinPool-1-worker-8 value: 9
>>doSth1: ForkJoinPool-1-worker-8
>>doSth2: ForkJoinPool-1-worker-13 value: 10
>>>forEach: ForkJoinPool-1-worker-13 value: 10
>>doSth1: ForkJoinPool-1-worker-13
>>doSth2: ForkJoinPool-1-worker-9 value: 11
>>>forEach: ForkJoinPool-1-worker-9 value: 11
>>doSth2: ForkJoinPool-1-worker-15 value: 12
>>>forEach: ForkJoinPool-1-worker-15 value: 12
>>doSth2: ForkJoinPool-1-worker-10 value: 13
>>>forEach: ForkJoinPool-1-worker-10 value: 13
>>doSth2: ForkJoinPool-1-worker-4 value: 14
>>>forEach: ForkJoinPool-1-worker-4 value: 14
>>doSth2: ForkJoinPool-1-worker-6 value: 15
>>>forEach: ForkJoinPool-1-worker-6 value: 15
>>doSth2: ForkJoinPool-1-worker-11 value: 16
>>>forEach: ForkJoinPool-1-worker-11 value: 16
>>doSth2: ForkJoinPool-1-worker-2 value: 17
>>>forEach: ForkJoinPool-1-worker-2 value: 17
>>doSth2: ForkJoinPool-1-worker-13 value: 18
>>>forEach: ForkJoinPool-1-worker-13 value: 18
>>doSth2: ForkJoinPool-1-worker-1 value: 19
>>>forEach: ForkJoinPool-1-worker-1 value: 19
>>doSth2: ForkJoinPool-1-worker-8 value: 20
>>>forEach: ForkJoinPool-1-worker-8 value: 20
Result: 20

当我将doSth（）方法更改为总是睡眠1秒而不是随机时间时，则结果将按顺序计算： 

>>doSth1: ForkJoinPool-1-worker-6
>>doSth1: ForkJoinPool-1-worker-1
>>doSth1: ForkJoinPool-1-worker-10
>>doSth1: ForkJoinPool-1-worker-2
>>doSth1: ForkJoinPool-1-worker-13
>>doSth1: ForkJoinPool-1-worker-15
>>doSth1: ForkJoinPool-1-worker-8
>>doSth1: ForkJoinPool-1-worker-11
>>doSth1: ForkJoinPool-1-worker-4
>>doSth1: ForkJoinPool-1-worker-9
>>doSth2: ForkJoinPool-1-worker-1 value: 1
>>doSth2: ForkJoinPool-1-worker-10 value: 2
>>doSth2: ForkJoinPool-1-worker-6 value: 3
>>>forEach: ForkJoinPool-1-worker-6 value: 3
>>>forEach: ForkJoinPool-1-worker-10 value: 2
>>>forEach: ForkJoinPool-1-worker-1 value: 1
>>doSth1: ForkJoinPool-1-worker-10
>>doSth1: ForkJoinPool-1-worker-6
>>doSth1: ForkJoinPool-1-worker-1
>>doSth2: ForkJoinPool-1-worker-15 value: 4
>>doSth2: ForkJoinPool-1-worker-9 value: 10
>>doSth2: ForkJoinPool-1-worker-8 value: 7
>>>forEach: ForkJoinPool-1-worker-8 value: 7
>>doSth1: ForkJoinPool-1-worker-8
>>doSth2: ForkJoinPool-1-worker-4 value: 9
>>>forEach: ForkJoinPool-1-worker-4 value: 9
>>doSth2: ForkJoinPool-1-worker-11 value: 8
>>doSth2: ForkJoinPool-1-worker-13 value: 6
>>doSth2: ForkJoinPool-1-worker-2 value: 5
>>>forEach: ForkJoinPool-1-worker-13 value: 6
>>>forEach: ForkJoinPool-1-worker-11 value: 8
>>doSth1: ForkJoinPool-1-worker-4
>>>forEach: ForkJoinPool-1-worker-9 value: 10
>>>forEach: ForkJoinPool-1-worker-15 value: 4
>>doSth1: ForkJoinPool-1-worker-9
>>doSth1: ForkJoinPool-1-worker-11
>>doSth1: ForkJoinPool-1-worker-13
>>>forEach: ForkJoinPool-1-worker-2 value: 5
>>doSth1: ForkJoinPool-1-worker-15
>>doSth1: ForkJoinPool-1-worker-2
>>doSth2: ForkJoinPool-1-worker-10 value: 12
>>>forEach: ForkJoinPool-1-worker-10 value: 12
>>doSth2: ForkJoinPool-1-worker-6 value: 11
>>doSth2: ForkJoinPool-1-worker-1 value: 13
>>>forEach: ForkJoinPool-1-worker-6 value: 11
>>>forEach: ForkJoinPool-1-worker-1 value: 13
>>doSth2: ForkJoinPool-1-worker-9 value: 15
>>doSth2: ForkJoinPool-1-worker-2 value: 20
>>>forEach: ForkJoinPool-1-worker-2 value: 20
>>doSth2: ForkJoinPool-1-worker-15 value: 19
>>>forEach: ForkJoinPool-1-worker-15 value: 19
>>doSth2: ForkJoinPool-1-worker-8 value: 14
>>doSth2: ForkJoinPool-1-worker-11 value: 17
>>doSth2: ForkJoinPool-1-worker-4 value: 16
>>doSth2: ForkJoinPool-1-worker-13 value: 18
>>>forEach: ForkJoinPool-1-worker-4 value: 16
>>>forEach: ForkJoinPool-1-worker-11 value: 17
>>>forEach: ForkJoinPool-1-worker-8 value: 14
>>>forEach: ForkJoinPool-1-worker-9 value: 15
>>>forEach: ForkJoinPool-1-worker-13 value: 18
Result: 20

是巧合还是对这种行为有解释？

2 个解决方案

解决方案1
 4 已采纳  2017-11-27 09:20:35

到那时，您正在执行sleep语句，没有已定义的顺序。 虽然通过IntStream.range创建的流具有已定义的遭遇顺序，但您通过忽略实际的int值将操作转换为无序操作。

产生可感知顺序的第一个操作是counter.incrementAndGet() 。 在此之前，哪个线程到达该点以及与之关联的流元素无关紧要。 它从AtomicInteger获得正确的数字。 之后，仅执行使用该编号的两个附加操作，使用该编号打印两个消息。 对于这两个不同的结果而言，重要的是这三个动作counter.incrementAndGet()和打印这两个消息是否被另一个线程截获。

我们可以轻松地将此方案删除 

AtomicInteger counter = new AtomicInteger();
ExecutorService es = Executors.newFixedThreadPool(20);
es.invokeAll(Collections.nCopies(20, () -> {
    out.println("1st: " + Thread.currentThread().getName());
    Thread.sleep(100 + new Random().nextInt(1000));
//    Thread.sleep(1000);
    int counterValue = counter.incrementAndGet();
    out.println("2nd: " + Thread.currentThread().getName() + " value: " + counterValue);
    out.println("3rd: " + Thread.currentThread().getName() + " value: " + counterValue);
    return null;
}));
es.shutdown();

请注意，对于invokeAll ，根本没有定义的排序，但是， invokeAll ，它并不重要。 这些任务在调用incrementAndGet()时最迟分配序列号。 行为与流示例相同。

虽然我总是强调并发并不意味着并行 ，但是由于未指定的执行时间和线程调度行为，当没有后台活动将不可预测的工作负载放在CPU上时，同时启动的短相同代码仍然很可能真正并行运行核心。

当所有线程并行运行时，它们同时out.println的内部同步，并且只有一个线程可以继续，其他线程被放入队列。 然后， synchronized的不公平性发挥作用。 任意线程将获胜，之后，任意线程将被重新安排。 这导致数字以随机顺序打印。

当你让线程休眠一段随机时间时，它们不再完全并行运行，从而增加了在不同时间到达print语句的机会，能够无条件地执行它们。 哪个线程首先到达这一点，是随机的，但由于它们在睡眠后得到了它们的数字，所以到达这一点的第一个线程将获得第一个，依此类推。

解决方案2
 1  2017-11-26 20:53:23

我怀疑这种行为的原因是在java.util.Random类（Java 8） java.util.Random调用nextInt 。 

>     protected int next(int bits) {
>         long oldseed, nextseed;
>         AtomicLong seed = this.seed;
>         do {
>             oldseed = seed.get();
>             nextseed = (oldseed * multiplier + addend) & mask;
>         } while (!seed.compareAndSet(oldseed, nextseed));
>         return (int)(nextseed >>> (48 - bits));
>     }

如果你不执行java.util.Random.nextInt(int) ，但预先计算随机值，结果仍然是不按顺序的 - 或者如果你创建自己的派生自java.util.Random的Random类并覆盖method protected int next(int bits)返回任何常量整数。 我试图预先计算随机值，如下所示： 

private static final int[] randomIntervals = IntStream.range(0, 20)
            .map(i -> new Random().nextInt(1000) + 100)
            .toArray();

然后在doSth方法中使用它： 

private static int doSth(AtomicInteger counter) {
    try {
        Thread.sleep(randomIntervals[counter.intValue()]);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }

    int counterValue = counter.incrementAndGet();
    out.println(">>doSth2: " + Thread.currentThread().getName() + " value: " + counterValue);

    return counterValue;
}

结果看起来类似于使用Thread.sleep(1000)的代码版本，如： 

>>doSth2: ForkJoinPool-1-worker-26 value: 8
>>doSth2: ForkJoinPool-1-worker-1 value: 4
>>>forEach: ForkJoinPool-1-worker-1 value: 4
>>doSth2: ForkJoinPool-1-worker-8 value: 7
>>>forEach: ForkJoinPool-1-worker-8 value: 7
>>doSth2: ForkJoinPool-1-worker-25 value: 1
>>>forEach: ForkJoinPool-1-worker-25 value: 1
>>doSth2: ForkJoinPool-1-worker-15 value: 6
>>>forEach: ForkJoinPool-1-worker-15 value: 6
>>doSth2: ForkJoinPool-1-worker-29 value: 5
>>>forEach: ForkJoinPool-1-worker-29 value: 5
>>doSth2: ForkJoinPool-1-worker-4 value: 2
>>>forEach: ForkJoinPool-1-worker-4 value: 2
>>doSth2: ForkJoinPool-1-worker-18 value: 3
>>>forEach: ForkJoinPool-1-worker-18 value: 3
>>>forEach: ForkJoinPool-1-worker-26 value: 8
>>doSth2: ForkJoinPool-1-worker-11 value: 9
>>>forEach: ForkJoinPool-1-worker-11 value: 9
>>doSth2: ForkJoinPool-1-worker-22 value: 10
>>>forEach: ForkJoinPool-1-worker-22 value: 10
>>doSth2: ForkJoinPool-1-worker-1 value: 11
>>doSth2: ForkJoinPool-1-worker-25 value: 12
>>>forEach: ForkJoinPool-1-worker-25 value: 12
>>doSth2: ForkJoinPool-1-worker-8 value: 13
>>>forEach: ForkJoinPool-1-worker-8 value: 13
>>>forEach: ForkJoinPool-1-worker-1 value: 11
>>doSth2: ForkJoinPool-1-worker-29 value: 14
>>>forEach: ForkJoinPool-1-worker-29 value: 14
>>doSth2: ForkJoinPool-1-worker-4 value: 15
>>>forEach: ForkJoinPool-1-worker-4 value: 15
>>doSth2: ForkJoinPool-1-worker-15 value: 16
>>>forEach: ForkJoinPool-1-worker-15 value: 16
>>doSth2: ForkJoinPool-1-worker-18 value: 17
>>>forEach: ForkJoinPool-1-worker-18 value: 17
>>doSth2: ForkJoinPool-1-worker-26 value: 18
>>>forEach: ForkJoinPool-1-worker-26 value: 18
>>doSth2: ForkJoinPool-1-worker-22 value: 19
>>>forEach: ForkJoinPool-1-worker-22 value: 19
>>doSth2: ForkJoinPool-1-worker-11 value: 20
>>>forEach: ForkJoinPool-1-worker-11 value: 20

另一个实验是调用java.util.Random的构造函数，但不调用nextInt 。 结果再次不按顺序： 

private static int doSth(AtomicInteger counter) {
    try {
        Random r = new Random();
        Thread.sleep(1000);
    } catch (InterruptedException e) {
        throw new RuntimeException(e);
    }

    int counterValue = counter.incrementAndGet();
    out.println(">>doSth2: " + Thread.currentThread().getName() + " value: " + counterValue);

    return counterValue;
}

暂无
暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 Java Stream API 惰性求值内部 Java 8 并行流和 ThreadLocal Java 8 | HashMap的并行流 Java 8并行流问题 Java并行流性能 流多个过滤器内部 Java Parallel Stream生成HashMap java流并行读写 Java并行流使用与不使用 Java 并行流 forEach 完成
相关标签
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM