Java REST Web服务中的线程

Question

I have a REST web service which is consuming too many CPU resources when the number of requests gets too high.

This is believed to be caused by a while() loop in the response generation, which normally takes a few milliseconds to complete, however under some circumstances it can take a few seconds.

The fix for this appears to be to use wait() and notify() according to this however I don't understand why this will reduce CPU usage?

Would this new thread be handled outside the web service, thereby freeing it up to handle more requests? Can someone please explain?

Thanks!

Jon.

Edit:

I may have found my own answer here

It seems that my code of result = get() constantly polls until there is a response, which consumes more CPU resources. By placing this in a thread, less resources are consumed.

Is this a correct understanding?

Answer 1

Is this a correct understanding?

A loop that continually polls something until there is a response is very wasteful. If you add a sleep between each poll, you reduce the CPU usage, but at the cost of reduced responsiveness for individual requests ... compared to what is achievable if you do it the right way.

Without knowing exactly what you are doing (what you are polling, and why) it is a bit difficult to say what the best solution is. But here are a couple of possible scenarios:

• If your web service is waiting for a response from an external service, then the simple solution is to just do a blocking read, and configure your web server with more worker threads.

• On the other hand, if your web service is waiting for a computation to complete, a new thread and wait / notify ... or one of the higher level synchronization classes ... may be the answer.

• If you need to handle a really large number of these blocking requests in parallel, that is going to require a lot of threads and hence a lot of memory, and other things. In that case you need to consider an web container that breaks the one-thread-per-request constraint. The latest version of the Servlet spec allows this, as do some of the alternative (non-Servlet) architectures.

---

FOLLOW UP

... I think the issue is your point 2, that the service is simply just waiting for the computation. So, by simply threading this computation will free up resources in the service?

If what you are describing is true, then running the computation in a different thread won't make it go much quicker. In fact, it could make it go slower.

The ultimate bottleneck is going to be CPU capacity, disc bandwidth and / or network bandwidth. Multi-threading is only going to make an individual request go faster if you can effectively / efficiently use 2 or more processors on the same request at the same time. It will only make your throughput better to the extent that it allows requests to run while others are waiting for external events; eg network responses to arrive, or file read/write operations to complete.

What I think you actually need to do is to figure out why the computation is taking so long and try and fix that:

• Are your database queries inefficient?
• Are fetching result-sets that are too large?
• Do you have a problem with your schemas?
• Poor choice of indexes?
• Or are you simply trying to do too much on a machine that is too small using the wrong kind of database?

There are various techniques for measuring performance of an application service and a database to determine where the bottlenecks are. (Start with a Google search ....)

Answer 2

Firstly, stop trying to guess why it's consuming so much CPU. Put some instrumentation in place and find out where the bottle-neck really is. Once you know where the bottle-neck is, try to to understand the root cause. 5 whys is a good technique to use when doing this.

Once you know the root cause, determine if it's solvable. eg if the algorithm is inefficient and there is an efficient algorithm is available then refactor the code to use the efficient algorithm. Don't forget that the root cause might be the server is under-powered and needs a bigger CPU.

Secondly, consider using a http cache to reduce the load on your server. How frequently does the data requested change and how fresh do you responses need to be? Use this to calculate a max-age; the length of time a HTTP cache could keep a response. It might be 1 min, a day, a week, etc. It really depends on what you are serving up. Once you have a max-age, have a look at the requests coming in. How many are for the same URI (or could be for the same URI if you tweeked it) within the max-age you've picked. If all the request URIs within a max-age period are unique then caching won't help you. If you're getting on average 2 requests for each URI in a max-age period, then a HTTP cache will half the load on you server. If it's 10 requests per URI, then a HTTP cache will reduce the server load by 90%.