What is the difference between thread per connection vs thread per request?

Question

Can you please explain the two methodologies which has been implemented in various servlet implementations:

1. Thread per connection
2. Thread per request

Which of the above two strategies scales better and why?

Answer 1

Which of the above two strategies scales better and why?

Thread-per-request scales better than thread-per-connection.

Java threads are rather expensive, typically using a 1Mb memory segment each, whether they are active or idle. If you give each connection its own thread, the thread will typically sit idle between successive requests on the connection. Ultimately the framework needs to either stop accepting new connections ('cos it can't create any more threads) or start disconnecting old connections (which leads to connection churn if / when the user wakes up).

HTTP connection requires significantly less resources than a thread stack, although there is a limit of 64K open connections per IP address, due to the way that TCP/IP works.

By contrast, in the thread-per-request model, the thread is only associated while a request is being processed. That usually means that the service needs fewer threads to handle the same number of users. And since threads use significant resources, that means that the service will be more scalable.

(And note that thread-per-request does not mean that the framework has to close the TCP connection between HTTP request ...)

---

Having said that, the thread-per-request model is not ideal when there are long pauses during the processing of each request. (And it is especially non-ideal when the service uses the comet approach which involves keeping the reply stream open for a long time.) To support this, the Servlet 3.0 spec provides an "asynchronous servlet" mechanism which allows a servlet's request method to suspend its association with the current request thread. This releases the thread to go and process another request.

If the web application can be designed to use the "asynchronous" mechanism, it is likely to be more scalable than either thread-per-request or thread-per-connection.

---

FOLLOWUP

Let's assume a single webpage with 1000 images. This results in 1001 HTTP requests. Further let's assume HTTP persistent connections is used. With the TPR strategy, this will result in 1001 thread pool management operations (TPMO). With the TPC strategy, this will result in 1 TPMO... Now depending on the actual costs for a single TPMO, I can imagine scenarios where TPC may scale better then TPR.

I think there are some things you haven't considered:

• The web browser faced with lots of URLs to fetch to complete a page may well open multiple connections.

• With TPC and persistent connections, the thread has to wait for the client to receive the response and send the next request. This wait time could be significant if the network latency is high.

• The server has no way of knowing when a given (persistent) connection can be closed. If the browser doesn't close it, it could "linger", tying down the TPC thread until the server times out the connection.

• The TPMO overheads are not huge, especially when you separate the pool overheads from the context switch overheads. (You need to do that, since TPC is going to incur context switches on a persistent connections; see above.)

My feeling is that these factors are likely to outweigh the TPMO saving of having one thread dedicated to each connection.

Answer 2

HTTP 1.1 - Has support for persistent connections which means more than one request/response can be received/sent using the same HTTP connection. So to run those requests received using the same connection in parallel a new Thread is created for each request.

HTTP 1.0 - In this version only one request was received using the connection and the connection was closed after sending the response. So only one thread was created for one connection.

Answer 3

Thread per connection is the Concept of reusing the same HTTP Connection from multiple requests ( keep-alive ).

Thread per request will create a thread for each request from a client .Server can create a number of threads as per request .

Answer 4

Thread per request will create a thread for each HTTP Request the server receives .

Thread per connection will reuse the same HTTP Connection from multiple requests(keep-alive) .AKA HTTP persistent connection but please note that this supported only from HTTP 1.1

Thread Per Request is faster as most web container use Thread Pooling.

The number of maximum parallel connections you should set on the number of cores on your server. More cores => more parallel threads .

See here how to configure... Tomcat 6: http://tomcat.apache.org/tomcat-6.0-doc/config/executor.html

Tomcat 7: http://tomcat.apache.org/tomcat-7.0-doc/config/executor.html

Example

Answer 5

Thread per request should be better, because it reuses threads, while some client may be idle. If you have a lot of concurrent users, they could be served with a less number of threads and having equal number of thread would be more expensive. There is also one more consideration - we do not know if user is still working with application, so we cant know when to destroy a thread. With a thread per request mechanism, we just use a thread pool.