Parallel processing threads on split in Apache Camel

Question

I am trying to split an incoming exchange in route based on some logic. Every split part would connect to an ssh server in a bean.

from("seda:compression")
    .routeId("Compressor")
    .split(beanExpression(new InstanceListParser(), "process"))
    .parallelProcessing()
        .to("bean:compressorClient?method=compress(${header.remoteHost},${header.dataCenter})")
    .end();

But it seems a maximum of 10 processes are being run in parallel.

I added the following code to increase the size of thread pool factory, but this does not help.

ThreadPoolProfile poolProfile = new ThreadPoolProfile("masterPoolProfile");
poolProfile.setMaxPoolSize(100);
poolProfile.setMaxQueueSize(100);
poolProfile.setPoolSize(50);
poolProfile.setKeepAliveTime(1L);
poolProfile.setTimeUnit(TimeUnit.MINUTES);

ThreadPoolFactory poolFactory = new DefaultThreadPoolFactory();
poolFactory.newThreadPool(poolProfile, Executors.defaultThreadFactory());

getContext().getExecutorServiceManager().setThreadPoolFactory(poolFactory);

Answer 1

You need to configure the splitter to use masterPoolProfile as the thread pool or configure the masterPoolProfile to be the default profile. The latter means all threads pools created using that API in Camel will use this profile as baseline.

Read more about this in the Camel docs: http://camel.apache.org/threading-model.html

Answer 2

This works for me:

ThreadPoolProfile poolProfile = new ThreadPoolProfile("masterPoolProfile");
poolProfile.setMaxPoolSize(100);
poolProfile.setMaxQueueSize(100);
poolProfile.setPoolSize(50);
poolProfile.setKeepAliveTime(1L);
poolProfile.setTimeUnit(TimeUnit.MINUTES);
getContext().getExecutorServiceManager().setDefaultThreadPoolProfile(poolProfile)