How to estimate or calculate the size of the ArrayBlockingQueue

Question

As title, in my module I had a blockingqueue to deliver my data. The data which server can produce is aa large number of logging information. In order to avoid affecting the performance of server , I wrote multi-thread clients to consume these data and persist them in data caches. Because the data can be produced hugely per mins,I became confused that how many sizes should I initialize my queue. And I knew that I can set my queue policy that if more data is produced , I can omit the overflow part. But how many size I created in the queue in order to hold these data as much as I can.

Could you give me some suggestion?As far as I know , it was related with my server JVM stack size & the single logging data in my JVM???

Answer 1

Make it "as large as is reasonable". For example, if you are OK with it consuming up to 1Gb of memory, then allocate its size to be 1Gb divided by the average number of bytes of the objects in the queue.

If I had to pick a "reasonable" number, I would start with 10000 . The reason is, if it grows to larger than that, then making it larger isn't a good idea and isn't going to help much, because clearly the logging requirement is outpacing your ability to log, so it's time to back off the clients.

"Tuning" through experimentation is usually the best approach, as it depends on the profile of your application:

• If there are highs and lows in your application's activity, then a larger queue will help "smooth out" the load on your server
• If your application has a relatively steady load, then a smaller queue is appropriate as a larger queue only delays the inevitable point when clients are blocked - you would be better to make it smaller and dedicate more resources (a couple more logging threads) to consuming the work.

Note also that a very large queue may impact garbage collection responsiveness to freeing up memory, as it has to traverse a much larger heap (all the objects in the queue) each time it runs, increasing the load on both CPU and memory.

You want to make the size as small as you can without impacting throughput and responsiveness too much. To asses this you'll need to set up a test server and hit it with a typical load to see what happens. Note that you'll probably need to hit it from multiple machines to put a realistic load on the server, as hitting it from one machine can limit the load due to the number of CPU cores and other resources on the test client machine.

To be frank, I'd just make the size 10000 and tune the number of worker threads rather than the queue size.

Answer 2

Contiguous writes to disk are reasonably fast (easily 20MB per second). Instead of storing data in RAM, you might be better off writing it to disk without worrying about memory requirements. Your clients then can read data from files instead of RAM.

To know size of java object, you could use any java profiler. YourKit is my favorite.

I think the real problem is not size of queue but what you want to do when things exceed your planned capacity. ArrayBlockingQueue will simply block your threads, which may or may not be the right thing to do. Your options typically are:

1) Block the threads (use ArrayBlockingQueue) based on memory committed for this purpose 2) Return error to the "layer above" and let that layer decide what to do...may be send error to the client 3) Can you throw away some data...say which was en queued long ago. 4) Start writing to disk, once you overflow RAM capacity.