如何避免 jdbc postgres 的多个连接

Question

I need to query a postgresql database based upon the result of the previuos query. Right now I have for each query a new DriverManager connection and this seems not like the best practise to me (I'm new to Java).

public static Integer item(DataModel params) {

    Psqlcon psql = new Psqlcon();
    
    try (Connection con = DriverManager.getConnection(psql.url, psql.user, psql.password);
         PreparedStatement pst = con.prepareStatement("SELECT * FROM table1 WHERE var=? "
        pst.setObject(1, params.getID());
       
        ResultSet find_id = pst.executeQuery();{

            while (find_id.next()) {

                id = find_id.getInt(1);
            }
            pst.close();
        }

    } catch (SQLException ex) {

        Logger lgr = Logger.getLogger(something.class.getName());
        lgr.log(Level.SEVERE, ex.getMessage(), ex);
    }

    try (Connection con = DriverManager.getConnection(psql.url, psql.user, psql.password);
             PreparedStatement pst = con.prepareStatement("SELECT something FROM table2 WHERE id=? "
            
        pst.setObject(1, id);
       
        ResultSet find_next = pst.executeQuery();{

            while (find_next.next()) {

                id2 = find_next.getInt(1);
            }
            pst.close();
        }
   
        ....on so on

This is an oversimplified example. The queries are actually a bit more complex. Wouldn't it be better to have one connection and close only the prepareStatement ? How do I accomplish to have only one connection?

Answer 1

Your question boils down to state tracking. Tons of your code across the entire breath of the project will want to do stuff with the DB, and DBs are fundamentally session oriented concepts. Thus, you will need to change your code flow – any code anywhere that needs to do anything with the DB can't just go: "Eh, fine, I'll go make a connection then". It needs to get an existing connection provided to it, because 'just make new connections every time' doesn't work. Not just because that is quite slow, but primarily because transactions are a thing.

This is where injection frameworks come in: They are all about having state that needs to exist in loads of places in the project. But, you can also do it 'by hand', so to speak: Pass a Connection object around as method parameter.

You have a few code smells in this code. These issues are leading you to this question.

Exception handling

'when an exception occurs, print some stuff to a log and just keeps going' is what your code does now. This is, hopefully now obvious given that I put it that way, a really bad idea. A method that fundamentally does DB things should be declared to throws SQLException . The entire 'code flow' (every method that ends up being called to perform a particular job) needs to be methods that ALL are declared as throws SQLException , at which point you can get rid of all your try/catch blocks. You'll be reducing your code LOC by quite a bit whilst you make the code better! Yay!

You want 'an exception' to do many things:

• Stop execution. You don't want code to continue when you're in a state you don't understand. SQLException has a ton of reasons for it, hence, you have no idea what happened. Carrying on is bad. One easy fix is to have the catch block rethrow something ( throw new RuntimeException("uncaught", e) - you should update your IDE templates, this is the correct default catch block content), but even better is to just let the SQLException happen.
• You want to save the relevant info as much as possible: The stack trace, the causal chain, the exact exception type, the message, and any special info that the exception object has - SQLException has lots of this, such as the postgres error state number which is very important, and is not always included in the message.

Just letting everything throw SQLException does all that - there will be very few places (all the top level calls) where you can write code that deals with it, instead of writing 83951 times lgr.log(Level.SEVERE) .

Connections are created every time

You don't want to do that - you want the connection to be passed along. In general you want a more robust setup to track transactions. All DB everythings are always in transactional form. Even if you have autocommit true (that's why it is called 'AutoCommit' and not 'transactionless'): That just means the DB assumes you meant to db.commit() after every query.

Even read queries are transactional. Imagine a bank that needs to hand out a bank guarantee for buyout by a partnership. To do so, it needs to sum up the balance of every member of the partnership. You could simply do:

int sum = 0;
for (Partner p : partners) {
  try (var pst = con.prepareStatement("SELECT balance FROM table2 WHERE id=?")) {
    ... 
    sum += rs.getInteger(1);
  }
}

return sum > requiredGuarantee;

However, the above code is subtly broken . Imagine that right in the middle of this code, one of the partners transfers a large sum to another partner. You so happen to read the partner transfering the cash out before the transfer, but read the balance of the receiving partner after it. Your code now has the wrong sum. It things the total amount in the accounts of all the partners is much higher than it really is, The bank gives the guarantee, the deal falls through. and the bank just lost 10 million dollars. Oops, Quite the bug? isn't it?

The solution is transactions. And the right transactionlevel. At the right level (SERIALIZABLE is the only level that can do this correctly), if you do all these SELECTs in a single transaction, the above situation cannot occur . Postgres will let you query and sum up as normal, and you may indeed get the wrong sum, but the .commit() call (and note that in this transaction we have not written anything at all yet!) will fail . That's your cue to start over.

Connections are at the same level as the transaction: If you want to run 2 transactions simultaneously, you must have 2 connections. A single connection is 'in' a transaction. Hence, passing the Connection object around is the same as passing the transaction around, and you must have transactions as the above code shows (imagine that the bank guarantee ends up being represented by INSERTing a new row in the 'guarantees' table. You want it all in a single transaction, so that after all that, you .commit() and postgres will throw you an exception (and that row you wrote never happened) if the summing wasn't consistent due to some other transaction having occurred in between that changed the balance of one or more partners.

That's the only correct way to write a 'consistent' data model.

For certain projects, the cost of consistency is too high. However, unless your name is literally google, you won't ever get there, and if you must have this, it's way more complicated than just picking a lower transaction isolation level. So, go for the consistency.

It's up to you if you want Connection con to be the first parameter of all your db-based methods, or if you want to create an object and have a field of type 'con' (but remember, they HAVE to be closed, so either the code that creates such an object has to use try-with-resources to close that connection, or your class that has a field Connection con needs to be AutoClosable , and has to be created with try-with-resources.) - or if you want to use an injection framework like spring or dagger or guice.

Use an abstraction

JDBC is primary designed to expose every feature that every relational DB system could possibly have, and to be a consistent 'bottom layer'. This means that as an API to directly program an app on, it's horrible . It's not meant for you.

Use a library that was meant for such a purpose. It'll be built on top of JDBC. You're doing the equivalent of writing direct machine code, effectively. That's rarely a good idea.

You can use JPA (Object Persistence Frameworks) like Hibernate, but that's a whole new can of worms. If you like SQL, the tools you are looking for are JOOQ or JDBI . These mean your complicated:

pst.setObject(1, id);
       
        ResultSet find_next = pst.executeQuery();{

            while (find_next.next()) {

                id2 = find_next.getInt(1);
            }
            pst.close();

all turn into simple one-liners.

connection pooling

Creating a connection object is really expensive. It's an inevitable cost you must pay if you want multiple simultaneous transactions, but you don't really want to pay it if you have multiple sequential ones. It depends a bit on the DB engine and JDBC driver (some even have a baked-in pooling system, some don't). But one easy solution is to add a layer of abstraction: Instead of asking the DriverManager to get you a connection, you can ask a connection pooler. It gives you a virtualized connection object that passes through all calls, except for close() , which doesn't actually close the connection, but simply invalidates the virtual object and passes the actual connection back into 'the pool'.

This is not necessary until you get quite a bit of traffic, but if you want to be fully ready, it's something you can look at. Libraries like HikariCP make this easy.