Remote database connection pooling from multiple servlets

Question

This is a multipart question, so please bear with me.

I'm developing a web based application with java servlets. There will be multiple application servers to load balance the system. These servers access a central remote database (none of them are in a LAN) to carry out the requests. One of the requirements of the system is to be able to dynamically add a new application server to the network and have it be able to connect to the database and start handling requests right away.

My questions:

• To setup a remote MySQL database server, I need to modify some configuration files to allow external connections. Which files are these and how can I modify them on the fly for new added servers whose IP address is not known until they are actually started up?
• For remote database access, should I be using connection pooling? Will the connection pooling be done individually for each servlet? How will the pool scale for newly added servers (more requests)?
• Each request to an application server goes through multiple separate database calls (example: finds and updates). Should I be using the same connection across these operations or releasing it every time?

Thank you

Answer 1

I don't know much about MySQL, so I won't answer the first part.

But yes, you should definitely use a connection pool. Connection setup time will be even worse for a distant database than a nearby one, so reusing connections is even more important.

Pools can be configured however you like, but are typically per-server, so they will be shared by all the servlets in a given server. If you configure a maximum size to the pool, then the total number of connections at the database will be proportional to the number of servers. This is something to be careful of - i've seen databases go down a few times because of inappropriately large pool sizes in large populations of clients.

Yes, you should reuse a single connection for the duration of each request (probably). For starters, you will have to do this if you want to use a single transaction across the whole request, which you almost certainly do. Aside from that, getting and releasing connections is not free, so reusing one amortizes the cost across several operations.

One caveat: holding on to a connection for the life of a request may increase the necessary pool size over grabbing and releasing for each operation. Usually, that's a good tradeoff. But if you are severely constrained by the number of connections, it might not be. It depends to some extent on how long your requests last, and how much database work they do, with shorter, busier requests making better use of a reused connection. If you're serving huge media streams which can be generated without reference to the database, it's possible your application doesn't fit this pattern.

A final point: if your database is a long way away, you will benefit more than usual from caching. Look hard at whether there is data you can usefully cache, or even store in a local database, at each server, to avoid trips to the central database.