通过不同的类同时处理对象列表的最佳方法是什么

Question

I am trying to process a object list at the same time with different classes, but I am not sure if I am heading in the right direction. I have read up about ExecutorService and Parallel stream, but not sure if it is the correct way.

So to provide an example:

I have a publisher that collects data and places it in a list. The publisher has multiple subscribers linked to it, which need to process the data and store it in a map. The map is then built up until all the data is processed and is then stored in a database table. Each subscriber has their own table that needs to be populate in some form with the data provided. What I am trying to accomplish is distributing the list to the subscribers at the same time and once all subscribers are finished processing the next set off data is supplied and so forth until all the data has been processed for a date range.

If anyone has some suggestions what I can look at, that would be awesome.

Answer 1

I'd start this way:

• Publisher
  • do not collect to a list, rather make it an Observable and have all the subscribers observe him.
  • On each batch, initialise a CountDownLatch to the batch size, and wait until it clears you.
• Subscriber
  • should observe events of the type they are interested in in a different thread (only one per type)
  • and on finishing with the handling of an event - they should notify the CDL of the Publisher.
• Publisher
  • after all the events are finished, and the CDL is released, should save the results to db
  • and then go on to the next batch

Now, all this is a suggestion, with multiple other possible solutions, and a very high-level one at that with many implementation details left unsaid. You should keep your eyes open as you work, and not be afraid to change routes if it is better.