I have a private Ethereum blockchain set up with 5 machines mining on it. The size of the block chain [number of blocks] are as of now, 300. The processing is done on back-end Java.
I need to run the following loop construct in a asynchronous manner. The bottleneck of the loop is during the execution of the following command:
EthBlock eb = web3.ethGetBlockByNumber(new DefaultBlockParameterNumber(BigInteger.valueOf(i)), true).send();
The command can also return a Completablefuture<EthBlock>
object by ending it with supplyAsync()
given here https://github.com/web3j/web3j#start-sending-requests Just calling supplyAync().get()
removes the parallelism aspect and makes it behave synchronously.
public void businessLogic() throws Exception {
recentBlocks = new ArrayList<EthBlock.Block>();
for (long i = 1; i <= 300000; i++) {
EthBlock eb = web3.ethGetBlockByNumber(new DefaultBlockParameterNumber(BigInteger.valueOf(i)), true).send();
if (eb == null || eb.getBlock() == null) {
continue;
}
EthBlock.Block block = eb.getBlock();
recentBlocks.add(block);
}
}
I not able to grasp the institution of translating the code into a way CompleteableFuture can operate on. Goal is to 'group' up multiple calls to web.ethGetBlockNumber(...).supplyAync()
into a collection and call them all at once to update an array which will get filled by EthBlock
objects ie recentBlocks
.
This is what I came up with:
public void businessLogic() throws Exception {
recentBlocks = new ArrayList<EthBlock.Block>();
List<CompleteableFuture> compFutures = new ArrayList<>();
for (long i = 0, i <= 300000, i++){
CompleteableFuture<EthBlock> compFuture = eb3.ethGetBlockByNumber(new DefaultBlockParameterNumber(BigInteger.valueOf(i)), true).sendAsync();
compFuture.thenAcceptAsync(eb -> // Doesn't look right
EthBlock.Block block = eb.getBlock();
recentBlock.add(block);)
compFutures.add(compFuture);
}
CompleteableFuture.allOf(compFutures).get();
}
Implementing IntStream
long start = System.nanoTime();
recentBlocks = IntStream.rangeClosed(0, 300_000)
.parallel()
.mapToObj(i -> {
try {
System.out.println("Current Thread -> " + Thread.currentThread());
return web3.ethGetBlockByNumber(new DefaultBlockParameterNumber(BigInteger.valueOf(i)), true).send();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
})
.filter(Objects::nonNull)
.map(EthBlock::getBlock)
.filter(Objects::nonNull)
.collect(Collectors.toList());
long stop = System.nanoTime();
System.out.println("Time Elapsed: " + TimeUnit.MICROSECONDS.convert(stop-start, TimeUnit.NANOSECONDS));
CompletableFuture contains an Override for get: get(long timeout, TimeUnit unit)
. You can use this to poll by making the get timeout if it does not return within a specific time.
You might be able to benefit from a parallel stream instead of relying on CompletableFuture
, assuming the order of the resulting List
isn't important:
IntStream.rangeClosed(0, 300_000)
.parallel()
.mapToObj(i -> web3.ethGetBlockByNumber(new DefaultBlockParameterNumber(BigInteger.valueOf(i)), true).send())
.filter(Objects::nonNull)
.map(EthBlock::getBlock)
.filter(Objects::nonNull)
.collect(Collectors.toList());
Because you stated that didn't help, let's try an ExecutorService
that utilizes a cached thread pool instead:
List<EthBlock.Block> blocks = Collections.synchronizedList(new ArrayList<>(300_000));
ExecutorService service = Executors.newCachedThreadPool();
for (int i = 0; i <= 300_000; i++) {
BigInteger number = BigInteger.valueOf(i);
service.execute(() -> {
EthBlock eb = web3.ethGetBlockByNumber(new DefaultBlockParameterNumber(number), true).send();
if (eb == null) {
return;
}
EthBlock.Block block = eb.getBlock();
if (block != null) {
blocks.add(block);
}
});
}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.