[英]Does Java have something similar to c++'s combinable?
I am using .parallelStream().forEach()
to do a bunch of writes to a DB, but would also like to keep a count of the rows I've written. 我正在使用.parallelStream().forEach()
对数据库进行大量写入操作,但也想保留我已写入的行数。
Right now, I have an java.util.concurrent.atomic.AtomicInteger
to keep track of the count, but I would like to use something similar to c++'s combinable<>
. 现在,我有一个java.util.concurrent.atomic.AtomicInteger
来跟踪计数,但是我想使用类似于c ++的combinable<>
。
In this article , there is an example of the use of combinable<>
: 在本文中 ,有一个使用combinable<>
的示例:
#include <iostream>
#include <cstdint>
#include <ppl.h>
using namespace Concurrency;
const int max_sum_item = 1000000000;
int main()
{
combinable<uint64_t> part_sums([] { return 0; });
parallel_for(0, max_sum_item,
[&part_sums] (int i)
{
part_sums.local() += i;
}
);
uint64_t result = part_sums.combine(std::plus<uint64_t>());
if (result != uint64_t(499999999500000000))
throw;
}
Is there an equivalent combinable<>
class in Java? Java中是否有一个等效的combinable<>
类?
Java code snippet: Java代码段:
AtomicInteger totalRows = new AtomicInteger(0);
...
myList.parallelStream().forEach(
... // write to db
totalRows.addAndGet(rowsWritten);
...
);
print(totalRows.get());
Looking for something like: 寻找类似的东西:
Combinable<int> totalRows = new Combinable<>(0);
...
myList.parallelStream().forEach(
... // write to db
totalRows = rowsWritten;
...
);
print(totalRows.combine());
EDIT: as per @zero323, the right tool in Spark would be an Accumulator
. 编辑:根据@ zero323,Spark中正确的工具将是一个Accumulator
。 I'm more interested in a multi-threaded case, but don't have a non-Spark example on hand atm. 我对多线程情况更感兴趣,但是atm上没有非Spark实例。
EDIT2: Updated example (and removing Spark references) EDIT2:更新示例(并删除Spark引用)
A right tool in Spark would be an Accumulator
: Spark中合适的工具将是一个Accumulator
:
Accumulator<Integer> accum = sc.accumulator(0);
myRDD.parallelStream().forEach(
accum.add(1);
);
accum.value();
Accumulators are write-only from a worker perspective and only can be read only by a driver. 从工作人员的角度来看,累加器是只写的,并且只能由驱动程序读取。 By default it supports only Long
, Double
and Float
but you can implement a custom AccumulatorParam
to support other types. 默认情况下,它仅支持Long
, Double
和Float
但是您可以实现自定义AccumulatorParam
以支持其他类型。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.