Invalid Jboss Datasources after redeploy

Question

I'm developing an application tha uses seven datasources (ds) for MySql databases. I'm pretty satisfied with the result when it is working. However i'm getting are datasource related problems when I redeploy my aplication.

Here is one of the seven ds I've configured in my my-application-ds.xml:

<xa-datasource>
            <jndi-name>jdbc/my_db_name</jndi-name>
            <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class>
            <xa-datasource-property name="URL">jdbc:mysql://192.168.26.2:3306/my_db_name</xa-datasource-property>
            <xa-datasource-property name="User">user</xa-datasource-property>
            <xa-datasource-property name="Password">password</xa-datasource-property>
            <min-pool-size>0</min-pool-size>
            <max-pool-size>5</max-pool-size>
            <idle-timeout-minutes>5</idle-timeout-minutes>
            <exception-sorter-class-name>
            com.mysql.jdbc.integration.jboss.ExtendedMysqlExceptionSorter
            </exception-sorter-class-name>
            <valid-connection-checker-class-name>
            com.mysql.jdbc.integration.jboss.MysqlValidConnectionChecker
            </valid-connection-checker-class-name>
            <track-connection-by-tx/>
</xa-datasource>

I think I've configured properly according with all i've found on the web about this.

1 . Deployed the application run a test OK

2 . Redeployed the application, run a test NOK

10:07:23,865 WARN [JBossManagedConnectionPool] Throwable while attempting to get a new connection: null

org.jboss.resource.JBossResourceException: Could not create connection; - nested throwable: (java.lang.NullPointerException)
at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.createManagedConnection(XAManagedConnectionFactory.java:144)
at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.createConnectionEventListener(InternalManagedConnectionPool.java:577)
at org.jboss.resource.connectionmanager.InternalManagedConnectionPool.getConnection(InternalManagedConnectionPool.java:262)
at org.jboss.resource.connectionmanager.JBossManagedConnectionPool$BasePool.getConnection(JBossManagedConnectionPool.java:500)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.getManagedConnection(BaseConnectionManager2.java:341)
at org.jboss.resource.connectionmanager.TxConnectionManager.getManagedConnection(TxConnectionManager.java:315)
at org.jboss.resource.connectionmanager.BaseConnectionManager2.allocateConnection(BaseConnectionManager2.java:396)
at org.jboss.resource.connectionmanager.BaseConnectionManager2$ConnectionManagerProxy.allocateConnection(BaseConnectionManager2.java:842)
at org.jboss.resource.adapter.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:88)
at org.hibernate.connection.DatasourceConnectionProvider.getConnection(DatasourceConnectionProvider.java:69)
at org.hibernate.cfg.SettingsFactory.buildSettings(SettingsFactory.java:84)
at org.hibernate.cfg.Configuration.buildSettings(Configuration.java:2073)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:1298)
at net.saphety.billing_core.dblayer.datasource.HibernateSession.configure(HibernateSession.java:58)
at net.saphety.billing_core.dblayer.datasource.HibernateSessionFactory.getHibernateSession(HibernateSessionFactory.java:25)
at net.saphety.billing_core.dblayer.datasource.HibDaoCRUD.<init>(HibDaoCRUD.java:35)
at net.saphety.billingcorewebapp.web.FilterHibernateControlSession.doFilter(FilterHibernateControlSession.java:43)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.NullPointerException
at com.mysql.jdbc.jdbc2.optional.MysqlDataSource.getConnection(MysqlDataSource.java:426)
at com.mysql.jdbc.jdbc2.optional.MysqlDataSource.getConnection(MysqlDataSource.java:137)
at com.mysql.jdbc.jdbc2.optional.MysqlDataSource.getConnection(MysqlDataSource.java:107)
at com.mysql.jdbc.jdbc2.optional.MysqlXADataSource.getXAConnection(MysqlXADataSource.java:47)
at org.jboss.resource.adapter.jdbc.xa.XAManagedConnectionFactory.createManagedConnection(XAManagedConnectionFactory.java:136)
... 34 more

After that hibernate complains about dialect...

org.hibernate.HibernateException: Hibernate Dialect must be explicitly set
at net.saphety.billing_core.dblayer.datasource.HibernateSession.configure(HibernateSession.java:62)
at net.saphety.billing_core.dblayer.datasource.HibernateSessionFactory.getHibernateSession(HibernateSessionFactory.java:25)
at net.saphety.billing_core.dblayer.datasource.HibDaoCRUD.<init>(HibDaoCRUD.java:35)
at net.saphety.billingcorewebapp.web.FilterHibernateControlSession.doFilter(FilterHibernateControlSession.java:43)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.jboss.web.tomcat.filters.ReplyHeaderFilter.doFilter(ReplyHeaderFilter.java:96)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.jboss.web.tomcat.security.SecurityAssociationValve.invoke(SecurityAssociationValve.java:179)
at org.jboss.web.tomcat.security.JaccContextValve.invoke(JaccContextValve.java:84)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.jboss.web.tomcat.service.jca.CachedConnectionValve.invoke(CachedConnectionValve.java:157)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:262)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:844)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:446)
at java.lang.Thread.run(Thread.java:619)

3 . Redeploy the *-ds.xml file and run a test NOK, nothing happens... In my browser I receive a blank response.

4 . Redeploy the application, all works fine again. :S

MySql server have enough connections available.

I have other project with data sources but with an oracle server and no problem there. So it has something to do with MySql but I can't figure out what.

Answer 1

These problems look like MySql problems, so you should probably be checking the logs there.

Caused by: java.lang.NullPointerException
    at com.mysql.jdbc.jdbc2.optional.MysqlDataSource.getConnection(MysqlDataSource.java:426)

indicates a failure deep in the connection with MySql. Given the amount of connections you have to the MySql instance. It could be that the connection is invalid, but JBoss is trying to reuse it.

Try changing your database connections to use:

  <min-pool-size>1</min-pool-size>

Or even zero, to not just grab pool connections on startup. Also see if you can configure MySql to allow more simultaneous connections. As it stands now just your JBoss deployment will take 35 connections on startup, and go to a maximum of 140. Do you really need so many connections and is the MySql instance configured to handle that many. If not, get these parameters more in line and your problems should get better.

Answer 2

A side answer. We sometimes find we need to bring down the server; delete the test, data, and work (log too if you don't wan't old ones clouding up) directories from your deployment; and bring the server back up. Most often this happens when we are running unstable code that crash with partial transactions completed and other messiness. These folders will be regenerated when JBoss starts up again. See this webpage for an overview of what these folders do.