如何使用线程池并行化单个石英作业？

Question

I have a quartz scheduled job in my Spring boot application to sends items from a large list to some webservices trough a method when fired in every 5 minutes.

I was wondering that my sending process (single send method) can be paralelized? What I want is for example when 10000 items list came from db and threads from thread pool will work simultanously to send all records in that list and job will be finished after all records sent.

What I've tried below code, I set a ThreadPoolTaskExecutor with a 5 threaded pool size. And yet when I executed and checked the job logs it says jobs is finished with seconds but it takes a minutes to send all data. It continue working correctly but Job seemed to finished in seconds. It probably says the job is finished after all threads set. This is something I avoid from because needings to know job execution time and logs.

@Autowired
MyService myService;

@NonTransactionalService 
public class MySenderService{

    ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
    taskExecutor.setCorePoolSize(5);
    taskExecutor.setMaxPoolSize(5);
    taskExecutor.initialize();

    public void sendAll(){
        List<Long> largeList = someMethod();
        largeList.stream().forEach(i -> {
            taskExecutor.execute(new Runnable() {
                @Override
                public void run() {
                    myService.send(i);
                }
            });
        }
    }
}

So, how can I run this send method with mutliple workers inside a single job?
Or else is it a good practise to set multiple identical jobs for sending same list through same method?

Answer 1

You can try using a Flux, IMHO is the solution that requires less code:

        ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
        taskExecutor.setCorePoolSize(5);
        taskExecutor.setMaxPoolSize(5);
        taskExecutor.initialize();

        List<Integer> largeList = someMethod();
        System.out.println(largeList);
        Flux.fromStream(largeList.stream())
                .parallel(5)
                .runOn(Schedulers.fromExecutor(taskExecutor))
                .subscribe( x -> { System.out.println(x);});

        taskExecutor.shutdown();

Don't forget to shutdown the TaskExecutor, but as @M. Deinum said, it would be best to have it created elsewhere and injected in your service.