简体   繁体   English

Java是否具有类似于c ++的可组合内容?

[英]Does Java have something similar to c++'s combinable?

I am using .parallelStream().forEach() to do a bunch of writes to a DB, but would also like to keep a count of the rows I've written. 我正在使用.parallelStream().forEach()对数据库进行大量写入操作,但也想保留我已写入的行数。

Right now, I have an java.util.concurrent.atomic.AtomicInteger to keep track of the count, but I would like to use something similar to c++'s combinable<> . 现在,我有一个java.util.concurrent.atomic.AtomicInteger来跟踪计数,但是我想使用类似于c ++的combinable<>

In this article , there is an example of the use of combinable<> : 本文中 ,有一个使用combinable<>的示例:

#include <iostream>
#include <cstdint>
#include <ppl.h>

using namespace Concurrency;

const int max_sum_item = 1000000000;

int main()
{
    combinable<uint64_t> part_sums([] { return 0; });

    parallel_for(0, max_sum_item,
        [&part_sums] (int i)
        {
            part_sums.local() += i;
        }
    );

    uint64_t result = part_sums.combine(std::plus<uint64_t>());

    if (result != uint64_t(499999999500000000))
        throw;
}

Is there an equivalent combinable<> class in Java? Java中是否有一个等效的combinable<>类?

Java code snippet: Java代码段:

AtomicInteger totalRows = new AtomicInteger(0);
...
myList.parallelStream().forEach(
    ... // write to db
    totalRows.addAndGet(rowsWritten);
    ...
);

print(totalRows.get());

Looking for something like: 寻找类似的东西:

Combinable<int> totalRows = new Combinable<>(0);
...
myList.parallelStream().forEach(
    ... // write to db
    totalRows = rowsWritten;
    ...
);

print(totalRows.combine());

EDIT: as per @zero323, the right tool in Spark would be an Accumulator . 编辑:根据@ zero323,Spark中正确的工具将是一个Accumulator I'm more interested in a multi-threaded case, but don't have a non-Spark example on hand atm. 我对多线程情况更感兴趣,但是atm上没有非Spark实例。

EDIT2: Updated example (and removing Spark references) EDIT2:更新示例(并删除Spark引用)

A right tool in Spark would be an Accumulator : Spark中合适的工具将是一个Accumulator

Accumulator<Integer> accum = sc.accumulator(0);

myRDD.parallelStream().forEach(
    accum.add(1);
);

accum.value();

Accumulators are write-only from a worker perspective and only can be read only by a driver. 从工作人员的角度来看,累加器是只写的,并且只能由驱动程序读取。 By default it supports only Long , Double and Float but you can implement a custom AccumulatorParam to support other types. 默认情况下,它仅支持LongDoubleFloat但是您可以实现自定义AccumulatorParam以支持其他类型。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM