Java Executor: Small tasks or big ones?

Question

Consider one big task which could be broken into hundreds of small, independently-runnable tasks. To be more specific, each small task is to send a light network request and decide upon the answer received from the server. These small tasks are not expected to take longer than a second, and involve a few servers in total.

I have in mind two approaches to implement this using the Executor framework, and I want to know which one's better and why.

1. Create a few, say 5 to 10 tasks each involving doing a bunch of send and receives.
2. Create a single task (Callable or Runnable) for each send & receive and schedule all of them (hundreds) to be run by the executor.

I'm sorry if my question shows that I'm lazy to test these and see for myself what's better (at least performance-wise). My question, while looking after an answer to this specific case, has a more general aspect. In situations like these when you want to use an executor to do all the scheduling and other stuff, is it better to create lots of small tasks or to group those into a less number of bigger tasks?

Answer 1

is it better to create lots of small tasks or to group those into a less number of bigger tasks?

It's hard to generalize with a question like this but in your case, I think it makes no sense to create some tasks that each do 5 to 10 things and then submit that to an executor service.

I would submit all of the work as a bunch of individual, small tasks to the ExecutorService . I think this would make the tasks much cleaner from an object/code perspective. They don't have to have a collection of requests/responses and can concentrate on making one request and processing one response.

One exception to this would be if you wanted to not send concurrent requests to a particular server. Then it would make sense to have a task do all of the requests/responses for a particular server and to submit a task for each server.

Answer 2

In general I would say smaller tasks are better, since they are like modules and can be combined in the desired fashion, even if requirements change.

However, if they can only be executed as a chunk you have a better overview with one big task.

Performance of course is also an issue. But this highly depends on the number of CPUs of your server/machine and the load of the whole system. Even if we'd knew, what exactly you are doing, you need to test to determine, what is more efficient. But do keep in mind, that the initialization and management of the tasks also takes some time and if your actions are really small, you might invest more time in the task management than in the actual execution.

Running all you (small) tasks in parallel might also prove to be difficult for the answering server, which need to process the requests. Keep in mind that web services or so usually also have a limit on how many parallel request they can serve.