Atomikos error prepared transaction “transaction id” with identifier does not exist

Question

I have spring boot app with Atomikos as transaction manager and I see this log:

ERROR: prepared transaction with identifier "1096044365_MTcyLjE3LjAuMy50bTAwNDAyMDAwMDE=_MTcyLjE3LjAuMy50bTQwMg==" does not exist

getting printed continuously.

After reading on Atomikos docs I understood that Atomikos has some crash recovery mechanism and I followed this http://fogbugz.atomikos.com/default4343.html?community.6.1918.14 but it didn't solve my issue.

My question is: why does Atomikos attempt to clean up a tx forever? When there's no chance of getting it to commit state.

Below is the stack trace:

org.postgresql.xa.PGXAException: Error rolling back prepared transaction at org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:419) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at com.atomikos.datasource.xa.XAResourceTransaction.rollback(XAResourceTransaction.java:636) ~[transactions-jta-3.9.3.jar!/:na] at com.atomikos.icatch.imp.RollbackMessage.send(RollbackMessage.java:70) [transactions-3.9.3.jar!/:na] at com.atomikos.icatch.imp.PropagationMessage.submit(PropagationMessage.java:83) [transactions-3.9.3.jar!/:na] at com.atomikos.icatch.imp.Propagator$PropagatorThread.run(Propagator.java:79) [transactions-3.9.3.jar!/:na] at com.atomikos.icatch.imp.Propagator.submitPropagationMessage(Propagator.java:58) [transactions-3.9.3.jar!/:na] at com.atomikos.icatch.imp.HeurHazardStateHandler.onTimeout(HeurHazardStateHandler.java:135) [transactions-3.9.3.jar!/:na] at com.atomikos.icatch.imp.CoordinatorImp.alarm(CoordinatorImp.java:933) [transactions-3.9.3.jar!/:na] at com.atomikos.timing.Poole dAlarmTimer.notifyListeners(PooledAlarmTimer.java:112) [atomikos-util-3.9.3.jar!/:na] at com.atomikos.timing.PooledAlarmTimer.run(PooledAlarmTimer.java:99) [atomikos-util-3.9.3.jar!/:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] Caused by: org.postgresql.util.PSQLException: ERROR: prepared transaction with identifier "1096044365_MTcyLjE3LjAuMy50bTAwNDAyMDAwMDE=_MTcyLjE3LjAuMy50bTQwMg==" does not exist at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org .postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:302) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:409) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] ... 12 common frames omitted

How to stop this retry mechanism

This is happening again after updating to 4.0.6

2018-01-22 20:42:52.247 ERROR [RequestId:] 1 --- [ Atomikos:51] cadatasource.xa.XAResourceTransaction : Heuristic termination of XAResourceTransaction: 3137322E31372E302E342E746D313531363637303131363034343030383435:3137322E31372E302E342E746D383435 with state HEUR_HAZARD 2018-01-22 20:42:52.247 ERROR [RequestId:] 1 --- [ Atomikos:51] com.atomikos.icatch.imp.CoordinatorImp : Local heuristic termination of coordinator 172.17.0.4.tm151667011604400845 with state HEUR_HAZARD 2018-01-22 20:42:52.247 ERROR [RequestId:] 1 --- [ Atomikos:51] com.atomikos.publish.EventPublisher : Transaction 172.17.0.4.tm151667011604400845 corrupted - check https://www.atomikos.com/Documentation/HowToHandleHeuristics to learn more 2018-01-22 20:42:52.420 WARN [RequestId:] 1 --- [ Atomikos:51] cadatasource.xa.XAResourceTransaction : XA resource 'postgres': rollback for XID '3137322E31372E302E342E746D313531363637303131363034343030383435:3137322E31372E302E342E746D383435' raised -3: the XA resource detected an intern al error

org.postgresql.xa.PGXAException: Error rolling back prepared transaction at org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:419) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at com.atomikos.datasource.xa.XAResourceTransaction.rollback(XAResourceTransaction.java:460) ~[transactions-jta-4.0.6.jar!/:na] at com.atomikos.icatch.imp.RollbackMessage.send(RollbackMessage.java:47) [transactions-4.0.6.jar!/:na] at com.atomikos.icatch.imp.RollbackMessage.send(RollbackMessage.java:20) [transactions-4.0.6.jar!/:na] at com.atomikos.icatch.imp.PropagationMessage.submit(PropagationMessage.java:67) [transactions-4.0.6.jar!/:na] at com.atomikos.icatch.imp.Propagator$PropagatorThread.run(Propagator.java:63) [transactions-4.0.6.jar!/:na] at com.atomikos.icatch.imp.Propagator.submitPropagationMessage(Propagator.java:42) [transactions-4.0.6.jar!/:na] at com.atomikos.icatch.imp.HeurHazardStateHandler.onTimeout(HeurHazardStateHandler.java:75) [transactions-4.0.6.jar!/:na] at com.atomikos.icatch.imp.Co ordinatorImp.alarm(CoordinatorImp.java:650) [transactions-4.0.6.jar!/:na] at com.atomikos.timing.PooledAlarmTimer.notifyListeners(PooledAlarmTimer.java:95) [atomikos-util-4.0.6.jar!/:na] at com.atomikos.timing.PooledAlarmTimer.run(PooledAlarmTimer.java:82) [atomikos-util-4.0.6.jar!/:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_151] Caused by: org.postgresql.util.PSQLException: ERROR: prepared transaction with identifier "1096044365_MTcyLjE3LjAuNC50bTE1MTY2NzAxMTYwNDQwMDg0NQ==_MTcyLjE3LjAuNC50bTg0NQ==" does not exist at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2103) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1836) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postg resql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:512) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:302) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] at org.postgresql.xa.PGXAConnection.rollback(PGXAConnection.java:409) ~[postgresql-9.1-901-1.jdbc4.jar!/:na] ... 13 common frames omitted

Answer 1

这看起来像是已在TransactionEssentials 4.0.6中修复的错误。

Answer 2

After digging into Atomikos source code and going through the stack traces. Below is the change I made(in Atomikos source code) to alarm method in CoordinatorImp.java and this solved my issue.

public void alarm ( AlarmTimer timer )
{
   try {
        boolean isHeuristic = stateHandler_ instanceof 
          HeurHazardStateHandler 
|| stateHandler_ instanceof HeurAbortedStateHandler
                                  || stateHandler_ instanceof HeurCommittedStateHandler || stateHandler_ instanceof HeurMixedStateHandler;
    if (!(isHeuristic)) {
     stateHandler_.onTimeout ();
    }
    } catch ( Exception e ) {
       LOGGER.logWarning( "Exception on timeout of coordinator " + root_ , e );
    }
   }