即使定义了超时，Postgres JDBC线程也会停留在java.net.SocketInputStream.socketRead0上

Question

We are using a Java app on a Wildfly 16 connected with a Postgres 9.4 database using EclipseLink. The Application Server is configured to use connection pooling. The app is indexing items to a Solr instance over night and from time to time we had problems with blocking threads. It seems that a database query isn't returning after all. We set a timeout for queries in our persistence unit, which doesn't seem to work. We added following entry in our persistence.xml:

<property name="javax.persistence.query.timeout" value="30000"/>

The system is using a Java 11 jre. We are using the docker wildfly image ( https://hub.docker.com/r/jboss/wildfly/dockerfile ) as a base, but have seen this problems with other Application Servers and Java versions as well.

The thread dump shows some details. The process is running multiple hours. We were expecting that a exception would be thrown after 30s. What are we missing?

"Indexer thread" #181 daemon prio=5 os_prio=0 cpu=3845600.45ms elapsed=124856.57s tid=0x000000000a51b800 nid=0x12d runnable  [0x00007fb51dcb9000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(java.base@11.0.4/Native Method)
        at java.net.SocketInputStream.socketRead(java.base@11.0.4/SocketInputStream.java:115)
        at java.net.SocketInputStream.read(java.base@11.0.4/SocketInputStream.java:168)
        at java.net.SocketInputStream.read(java.base@11.0.4/SocketInputStream.java:140)
        at org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:146)
        at org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:115)
        at org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:71)
        at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:283)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1719)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:173)
        - locked  (a org.postgresql.core.v3.QueryExecutorImpl)
        at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:622)
        at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:472)
        at org.postgresql.jdbc.PgStatement.executeUpdate(PgStatement.java:429)
        at jdk.internal.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@11.0.4/DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(java.base@11.0.4/Method.java:566)
        at org.postgresql.ds.PGPooledConnection$StatementHandler.invoke(PGPooledConnection.java:466)
        at com.sun.proxy.$Proxy69.executeUpdate(Unknown Source)
        at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:537)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:898)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:970)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:640)
        at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:567)
        at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2096)
        at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:311)
        at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:275)
        at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:261)
        at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.updateObject(DatasourceCallQueryMechanism.java:832)
        at org.eclipse.persistence.internal.queries.StatementQueryMechanism.updateObject(StatementQueryMechanism.java:437)
        at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.updateObjectForWriteWithChangeSet(DatabaseQueryMechanism.java:1093)
        at org.eclipse.persistence.queries.UpdateObjectQuery.executeCommitWithChangeSet(UpdateObjectQuery.java:86)
        at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.executeWriteWithChangeSet(DatabaseQueryMechanism.java:316)
        at org.eclipse.persistence.queries.WriteObjectQuery.executeDatabaseQuery(WriteObjectQuery.java:60)
        at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:914)
        at org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:813)
        at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWorkObjectLevelModifyQuery(ObjectLevelModifyQuery.java:110)
        at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWork(ObjectLevelModifyQuery.java:87)
        at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2981)
        at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1895)
        at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1877)
        at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1827)
        at org.eclipse.persistence.internal.sessions.CommitManager.commitChangedObjectsForClassWithChangeSet(CommitManager.java:275)
        at org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjectsWithChangeSet(CommitManager.java:133)
        at org.eclipse.persistence.internal.sessions.AbstractSession.writeAllObjectsWithChangeSet(AbstractSession.java:4387)
        at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabase(UnitOfWorkImpl.java:1493)
        at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabaseWithChangeSet(UnitOfWorkImpl.java:1583)
        at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.commitRootUnitOfWork(RepeatableWriteUnitOfWork.java:280)
        at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitAndResume(UnitOfWorkImpl.java:1220)
        at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commit(EntityTransactionImpl.java:136)
        at com.freiheit.vrms.persistence.SystemObjectPersistenceService.createOrUpdateNextIndexerRun(SystemObjectPersistenceService.java:2134)
        at com.freiheit.vrms.service.search.SearchService.attributeIndexFields(SearchService.java:322)
        at com.freiheit.vrms.service.search.SearchService.indexObjectInternal(SearchService.java:848)
        at com.freiheit.vrms.service.search.SearchService.indexObjectInForeground(SearchService.java:752)
        - locked  (a java.lang.Object)
        at com.freiheit.vrms.service.search.SearchService.indexObjectInForeground(SearchService.java:746)
        at com.freiheit.vrms.service.search.IndexJob$JobType$2.execute(IndexJob.java:60)
        at com.freiheit.vrms.service.search.IndexJob.execute(IndexJob.java:104)
        at com.freiheit.vrms.service.search.IndexQueue.handleMessage(IndexQueue.java:243)
        at com.freiheit.vrms.service.search.IndexQueue.run(IndexQueue.java:155)
        at java.lang.Thread.run(java.base@11.0.4/Thread.java:834)

   Locked ownable synchronizers:
        -  (a java.util.concurrent.locks.ReentrantLock$FairSync)

EDIT: I would like to add some additional code, which causes the thread to freeze.

final EntityTransaction transaction = em.getTransaction();
transaction.begin();
updateNextIndexerRun( em, id, nextIndexerRun );
transaction.commit();

As shown above the thread stucked at the line transaction.commit() . This snippet is called thousands of times a day and isn't making any trouble most of the time. the table in which the data will be persisted has ~20000 entries at a time. The method updateNextIndexerRun is defined this way:

private void updateNextIndexerRun( final EntityManager em, final Long id, @Nullable final Calendar newNextIndexerRun ) {

    final NextIndexerRunDBBean oldNextIndexerRun = getNextIndexerRun( em, id );

    if ( newNextIndexerRun == null && oldNextIndexerRun != null ) {
        em.remove( oldNextIndexerRun );
    } else if ( newNextIndexerRun != null ) {
        if ( oldNextIndexerRun == null ) {
            em.persist( new NextIndexerRunDBBean( id, newNextIndexerRun ) );
        } else {
            em.merge( new NextIndexerRunDBBean( id, newNextIndexerRun ) );
        }
    }
}

Answer 1

The query itself is run under 30 seconds, so the timeout isn't being applied. Now from here we see

the JPQL query above will time out after 50 milliseconds unless the result set is being fetched prior to the timeout threshold.

You're already reading the resultset, but it's either too large or there's a problem reading it fast enough.