Spring Boot Web服务JDBC偶尔无法获得连接

Question

We are having a problem with our Spring Boot web applications, but we cannot narrow it down, nor can we reliably reproduce it. Occasionally, our application will fail to acquire a JDBC connection, and will just hang for 15 minutes. After 15 minutes, a connection to the database will be acquired, the process will run smoothly and complete (though it fails to return a response with a broken pipe error caused from Apache timing out and closing the socket long before this process completes). The application logs at this time show:

DEBUG osjdbc.datasource.DataSourceUtils - Fetching JDBC Connection from DataSource

then nothing until the application picks up 15 minutes later. MYSQL logs show nothing here. In the mean time, other transactions run through this application just fine, db connections made, no problem. But this one thread is locked up.

We are using spring-boot-starter-jdbc and java 1.8 connecting to a Maria 10.0 db. Here is our application.properties as pertaining to the datasource setup:

spring.datasource.url=jdbc:mysql://ourDatabase
spring.datasource.username=AUser
spring.datasource.password=aSecretPassword
spring.datasource.driver-class-name=com.mysql.jdbc.Driver

Finally, we did receive an error stacktrace that looks like it is related:

2017-12-21 09:36:52,913 70038378 [ajp-bio-8009-exec-1] DEBUG o.a.t.jdbc.pool.PooledConnection  - Unable to validate object:
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 3,790,114 milliseconds ago.  The last packet sent successfully to the server was 929,256 milliseconds ago.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3559)
at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3459)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3900)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2486)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2444)
at com.mysql.jdbc.StatementImpl.executeInternal(StatementImpl.java:845)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:745)
at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:532)
at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:443)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:802)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:651)
at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:198)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:132)

Answer 1

By changing a couple lines and adding a new line to application.properties

spring.datasource.url=jdbc:mariadb://ourDatabase

spring.datasource.driver-class-name=org.mariadb.jdbc.Driver

spring.datasource.type=org.apache.tomcat.jdbc.pool.DataSource

We produced a similar exception, with the rest of the results the same as stated above.

2017-12-21 16:56:41,959 4475051 [ajp-bio-8009-exec-2] DEBUG o.a.t.jdbc.pool.PooledConnection  - Unable to validate object:
java.sql.SQLNonTransientConnectionException: (conn=1076254) Connection timed out (Read failed)
        at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:175)
        at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:110)
        at org.mariadb.jdbc.MariaDbStatement.executeExceptionEpilogue(MariaDbStatement.java:235)
        at org.mariadb.jdbc.MariaDbStatement.executeInternal(MariaDbStatement.java:332)
        at org.mariadb.jdbc.MariaDbStatement.execute(MariaDbStatement.java:383)
        at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:532)
        at org.apache.tomcat.jdbc.pool.PooledConnection.validate(PooledConnection.java:443)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:802)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:651)
        at org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:198)
        at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:132)
        at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111)
        at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:394)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:474)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:484)
        at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:494)
        at org.springframework.jdbc.core.JdbcTemplate.queryForObject(JdbcTemplate.java:500)

This exception falls between

Fetching JDBC Connection from DataSource

and

 Returning JDBC Connection to DataSource 

Answer 2

Ultimately the solution worked for us, was to specifically designate to use the Hikari Datasource in our application.properties . Once we defined Hikari as our datasource, instead of the default Tomcat and mysqlDriver, the problem stopped and we haven't seen it since.