How can I make this code more concurrent?

Question

I have a piece of code which is similar to the following:

final int THREADS = 11;  
BlockingQueue<Future<Long>> futureQueue = new ArrayBlockingQueue<Future<Long>>(THREADS);  
for (int i = 0; i < end; i++, count++) {  
    futureQueue.put(executor.submit(MyRunnable));  
}   
//Use queued results

How could I refactor this to make it more concurrent ? Are there any subtleties I am overseeing here?

UPDATE:
Each Runnable is supposed to send a large amount of HTTP request to a server for stress testing.Am I on right track?

Answer 1

I would use

static final int THREADS = Runnable.getRuntime().availableProcesses();  

ExecutorService service = Executors.newFixedThreadPool(THREADS);

List<Future<Long>> futureQueue = new ArrayList<Future<Long>>(end);  
for (int i = 0; i < end; i++)
    futureQueue.add(executor.submit(new MyRunnable()));  

You are using a bounded queue and if end > THREADS it will just stop.

Each Runnable is supposed to send a large amount of HTTP request to a server for stress testing.Am I on right track?

In that case I would use the following as your code is IO rather than CPU bound.

ExecutorService service = Executors.newCachedThreadPool();

While you might benefit from using NIO if you had more than 1000 threads, this would only make your load tester more efficient, but it would make the code much more complicated. (If you think this is hard, writing efficient and correct Selector code is much harder)

Run the tester on more than one machine would make much more difference.

Answer 2

Using threads doesn't work well in your case. Thread pools work well when you have a CPU intensive job. In your case, you have an IO intensive job - it's not bound by the number of CPUs that you have but by the number of network packets you can send.

In this case, the classes in NIO are your friend. Create hundreds of connections and use NIO selectors to see which one is ready to receive more data.

Using this approach, you don't need threads at all; one CPU core is more than enough to fill even a GBit Ethernet connection (~100MB/s).

[EDIT] Of course, you could create hundreds of threads to try to fill the IO channel. But this has some drawbacks:

1. Threads are managed by the OS (or a small helper library). They need memory and each time a thread is switched, the CPU will have to save its state and flush its caches.
2. If a thread does only a small amount of work, thread switching can be more expensive than doing the work.
3. When you use threads, you get all the usual thread synchronization issues.
4. There is no simple way to make sure you have the right amount of threads. If there are too few threads, the IO channel won't be used optimally. If there are too many threads, the channel won't be used optimally because the threads will fight for access for it. In both cases, you can't change this after starting your tests. The system doesn't adapt to the needs.

For tasks like this, a framework like Akka is much better suited because it avoids all these issues and it's more simple to use than threads.