Confusion with DBCP connection pooling and Threads

Question

I am trying to get Multi-Threading working in my Java web application and it seems like no matter what I try I run into some sort of issues with connection pooling.

My current process is that I have am looping through all my departments and processing them, which finally generates a display. This takes time, so I want to spawn a thread for each department and have them process concurrently.

After alot of time of first figuring out how to get my hibernate session to stay open in a thread to prevent the lazy initializion loading errors, I finally had the solution of creating a Spring bean of my Thread class, and creating a new instance of that bean for each thread. I have tried 2 different versions

1) I directly inject the DAO classes into the Bean. FAILED - After loading the page a few times I would get "Cannot get a connection, pool error Timeout waiting for idle object" for each thread and the app would crash.

2) Ok so then I tried injecting the spring SessionFactory into my bean, then create new instances of my DAO and set it with the SessionFactory . My DAO objects all extend HibernateDaoSupport . FAILED - After loading the page a few times I would get "too many connections" error messages.

Now what I am confused about is that my SessionFactory bean is a singleton which I understand to mean that it is a single Object that is shared throughout the Spring container. If that is true then why does it appear like each Thread is creating a new connection when it should just be sharing that single instance? It appears that all my connection pool is getting filled up and I don't understand why. Once the threads are done, all the connections that were created should be released but there not. I even tried running a close() operation on the injected SessionFactory but that has no effect. I tried limiting how many threads run concurrently at 5, hoping that would cause not so many connections to be created at one time but no luck.

I am obviously doing something wrong but I am not sure what. Am I taking the wrong approach entirely in trying to get my hibernate Session into my Thread s? Am I somehow not managing my connection pool properly? any ideas would be greatly appreciated!

More info I thought of: My process creates about 25 threads total, which are run 5 at a time. I am able to refresh my page about 3 times before I start getting the errors. So apparently each refresh creates and holds on to a bunch of connections.

Here is my spring config file:

<bean id="mainDataSource" class="org.apache.commons.dbcp.BasicDataSource"
    destroy-method="close">

....

<!-- Purposely put big values in here trying to figure this issue out 
since my standard smaller values didn't help.  These big values Fail just the same -->

    <property name="maxActive"><value>500</value></property>
    <property name="maxIdle"><value>500</value></property>
    <property name="minIdle"><value>500</value></property>
    <property name="maxWait"><value>5000</value></property>
    <property name="removeAbandoned"><value>true</value></property>
    <property name="validationQuery"><value>select 0</value></property>
</bean>

<!--Failed attempt #1 -->
<bean id="threadBean" class="controller.manager.ManageApprovedFunds$TestThread" scope="prototype">
        <property name="dao"><ref bean="dao" /></property>
    </bean>

<!--Failed attempt #2 -->
<bean id="threadBean" class="manager.ManageApprovedFunds$TestThread" scope="prototype">
        <property name="sessionFactory"><ref bean="sessionFactory" /></property>
    </bean>

Java code:

ExecutorService taskExecutor = Executors.newFixedThreadPool(5);

for(Department dept : getDepartments()) {
    TestThread t = (TestThread)springContext.getBean("threadBean");
    t.init(dept, form, getSelectedYear());
    taskExecutor.submit(t);
}

taskExecutor.shutdown();

public static class TestThread extends HibernateDaoSupport implements Runnable, ApplicationContextAware {
        private ApplicationContext appContext;

        @Override
        public void setApplicationContext(ApplicationContext arg0)
                throws BeansException {
            this.appContext = arg0;
        }

        @Override
        public void run() {
            try {
                MyDAO dao = new MyDAO();
                dao.setSessionFactory(getSessionFactory());

                //SOME READ OPERATIONS

                getSessionFactory().close();
            } catch (Exception e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }       
    }

Answer 1

Basically you shouldn't try to share a single hibernate Session between threads. Hibernate sessions, nor the entity objects, are threadsafe and that can lead to some suprising problems. Here and [here]3[] is a small but nice read there is also some value in the comments. Basically don't try to share a Session between threads.

There is also another problem as the whole transaction management is based around ThreadLocal objects, the same goes for obtaining the current session and underlying JDBC connection. Now trying to spawn threads will lead to suprising problems, one of them would be connection pool starvation. (Note: don't open too many connections, more information here ).

Next to not to opening to much connections you should be aware of starting to many threads. A Thread is bound to a cpu (or core if you have multiple cores). Adding to many threads might lead to heavy sharing of a single cpu/core between to many threads. This can kill your performance instead of increasing it.

In short IMHO your approach is wrong, each thread should simply read the entity it cares about, do its thing and commit the transaction. I would suggest using something like Spring Batch for this instead of inventing your own mechanism. (Although if it is simple enough I would probably go for it).

Answer 2

A database connection is not associated with the SessionFactory but with the Session . To get your handling correct you have to take care of the following:

• Only create one instance of SessionFactory (per persistence context) - but you are doing this already
• Don't close() the SessionFactory - it's lifetime should end when the application dies - that is at undeployment or server-shutdown.
• Make sure to always close() your Session - you use one database connection per open Session and if these don't get closed you are leaking connections.