Using Hibernate + Spring with 3 Transactional Databases

Question

Greetings from Ecuador:

Currently I have created a project in which I must perform operations on 3 different databases. For this purpose I decided to use Hibernate ORM 5.2.7, Spring Framework 4.3.6 and other libraries for implementation of connection pools among others. For the implementation of the configuration of the context of Spring support me in annotations which I show below:

@Configuration
@ComponentScan("fttg.*")
@EnableTransactionManagement
@EnableScheduling
@PropertySources({
       @PropertySource("classpath:application.properties"),
       @PropertySource("classpath:schedule.properties")
})
public class ApplicationConfig {

      @Autowired
      private Environment environment;

@Bean(destroyMethod = "close")
public BasicDataSource dataSourceBitacora() {
    BasicDataSource dataSource = new BasicDataSource();
    dataSource.setDriverClassName(environment.getRequiredProperty("postgres.jdbc.driver"));
    dataSource.setUrl(environment.getRequiredProperty("bitacora.jdbc.url"));
    dataSource.setUsername(environment.getRequiredProperty("bitacora.jdbc.username"));
    dataSource.setPassword(environment.getRequiredProperty("bitacora.jdbc.password"));
    dataSource.setPoolPreparedStatements(true);
    dataSource.setInitialSize(4);
    dataSource.setMaxTotal(4);
    dataSource.setMaxIdle(2);
    dataSource.setMinIdle(1);
    dataSource.setDefaultAutoCommit(Boolean.FALSE);
    return dataSource;
}

@Bean(destroyMethod = "close")
public BasicDataSource dataSourceFacturacion() {
    BasicDataSource dataSource = new BasicDataSource();
    dataSource.setDriverClassName(environment.getRequiredProperty("postgres.jdbc.driver"));
    dataSource.setUrl(environment.getRequiredProperty("facturacion.jdbc.url"));
    dataSource.setUsername(environment.getRequiredProperty("facturacion.jdbc.username"));
    dataSource.setPassword(environment.getRequiredProperty("facturacion.jdbc.password"));
    dataSource.setPoolPreparedStatements(true);
    dataSource.setInitialSize(1);
    dataSource.setMaxTotal(4);
    dataSource.setDefaultAutoCommit(Boolean.FALSE);
    return dataSource;
}

@Bean(destroyMethod = "close")
public BasicDataSource dataSourceSietab() {
    BasicDataSource dataSource = new BasicDataSource();
    dataSource.setDriverClassName(environment.getRequiredProperty("postgres.jdbc.driver"));
    dataSource.setUrl(environment.getRequiredProperty("sietab.jdbc.url"));
    dataSource.setUsername(environment.getRequiredProperty("sietab.jdbc.username"));
    dataSource.setPassword(environment.getRequiredProperty("sietab.jdbc.password"));
    dataSource.setPoolPreparedStatements(true);
    dataSource.setInitialSize(1);
    dataSource.setMaxTotal(2);
    dataSource.setDefaultAutoCommit(Boolean.FALSE);
    return dataSource;
}

@Bean
public LocalSessionFactoryBean sessionFactoryBitacora() {
    LocalSessionFactoryBean sessionFactory = new LocalSessionFactoryBean();
    sessionFactory.setDataSource(dataSourceBitacora());
    sessionFactory.setPackagesToScan(environment.getRequiredProperty("bitacora.sessionFactory.packagesToScan"));
    Properties properties = new Properties();
    properties.put("hibernate.dialect", environment.getRequiredProperty("postgres.hibernate.dialect"));
    properties.put("hibernate.show_sql", environment.getRequiredProperty("hibernate.show_sql"));
    sessionFactory.setHibernateProperties(properties);
    return sessionFactory;
}

@Bean
public LocalSessionFactoryBean sessionFactoryFacturacion() {
    LocalSessionFactoryBean sessionFactory = new LocalSessionFactoryBean();
    sessionFactory.setDataSource(dataSourceFacturacion());
    sessionFactory.setPackagesToScan(environment.getRequiredProperty("facturacion.sessionFactory.packagesToScan"));
    Properties properties = new Properties();
    properties.put("hibernate.dialect", environment.getRequiredProperty("postgres.hibernate.dialect"));
    properties.put("hibernate.show_sql", environment.getRequiredProperty("hibernate.show_sql"));
    sessionFactory.setHibernateProperties(properties);
    return sessionFactory;
}

@Bean
public LocalSessionFactoryBean sessionFactorySietab() {
    LocalSessionFactoryBean sessionFactory = new LocalSessionFactoryBean();
    sessionFactory.setDataSource(dataSourceSietab());
    sessionFactory.setPackagesToScan(environment.getRequiredProperty("sietab.sessionFactory.packagesToScan"));
    Properties properties = new Properties();
    properties.put("hibernate.dialect", environment.getRequiredProperty("postgres.hibernate.dialect"));
    properties.put("hibernate.show_sql", environment.getRequiredProperty("hibernate.show_sql"));
    sessionFactory.setHibernateProperties(properties);
    return sessionFactory;
}

@Bean
public HibernateTransactionManager transactionManagerBitacora() {
    HibernateTransactionManager txManager = new HibernateTransactionManager();
    txManager.setSessionFactory(sessionFactoryBitacora().getObject());
    return txManager;
}

@Bean
public HibernateTransactionManager transactionManagerFacturacion() {
    HibernateTransactionManager txManager = new HibernateTransactionManager();
    txManager.setSessionFactory(sessionFactoryFacturacion().getObject());
    return txManager;
}

@Bean
public HibernateTransactionManager transactionManagerSietab() {
    HibernateTransactionManager txManager = new HibernateTransactionManager();
    txManager.setSessionFactory(sessionFactorySietab().getObject());
    return txManager;
}
}

The DAOS configuration is the same for all objects in the database:

@Repository
public class BitacoraFacturasDetalleDao extends GenericDaoImpl<BitacoraFacturasDetalle, Integer>{

private final static Logger LOGGER = Logger.getLogger(BitacoraFacturasDetalleDao.class);

@Qualifier("sessionFactoryBitacora")
@Autowired
private SessionFactory sessionFactory;

public BitacoraFacturasDetalleDao() {
    super(BitacoraFacturasDetalle.class);
}

public BitacoraFacturasDetalle findByEstablecimientoAndPuntoEmisionAndSecuencial(String establecimiento, String puntoEmision, String secuencial) {
    LOGGER.info("evento findByEstablecimientoAndPuntoEmisionAndSecuencial");
    BitacoraFacturasDetalle ret = (BitacoraFacturasDetalle) getCurrentSession().createNamedQuery("BitacoraFacturasDetalle.findByEstablecimientoAndPuntoEmisionAndSecuencial").setParameter("establecimiento", establecimiento).setParameter("puntoEmision", puntoEmision).setParameter("secuencial", secuencial).uniqueResult();
    return ret;
}

   @Override
   protected Session getCurrentSession() {
       return this.sessionFactory.getCurrentSession();
   }

}

Transactional objects are implemented as follows:

@Service("facturasService")
@Transactional(value="transactionManagerFacturacion", readOnly = false)
public class FacturasServiceImpl implements FacturasService, Serializable {

private static final long serialVersionUID = 1L;
private final static Logger LOGGER =        Logger.getLogger(FacturasServiceImpl.class);

@Autowired
private FacturasCabeceraDao facturasCabeceraDao;

@Override
public boolean save(FacturasCabecera factura) {
    LOGGER.info("evento save");
    return facturasCabeceraDao.save(factura);
 }

}

@Service("bitacoraFacturasDetalleService")
@Transactional(readOnly = false, value = "transactionManagerBitacora")
public class BitacoraFacturasDetalleServiceImpl implements BitacoraFacturasDetalleService, Serializable {

private static final long serialVersionUID = 1L;
private final static Logger LOGGER = Logger.getLogger(BitacoraFacturasDetalleServiceImpl.class);

@Autowired
private BitacoraFacturasDetalleDao bitacoraFacturasDetalleDao;

@Override
public boolean save(BitacoraFacturasDetalle b) {
    LOGGER.info("evento save");
    return bitacoraFacturasDetalleDao.save(b);
}

@Override
public boolean edit(BitacoraFacturasDetalle b) {
    LOGGER.info("evento edit");
    return bitacoraFacturasDetalleDao.edit(b);
}

@Override
@Transactional(readOnly = true, value = "transactionManagerBitacora")
public BitacoraFacturasDetalle findByEstablecimientoAndPuntoEmisionAndSecuencial(String establecimiento, String puntoEmision, String secuencial) {
    LOGGER.info("evento findByEstablecimientoAndPuntoEmisionAndSecuencial");
    return   bitacoraFacturasDetalleDao.findByEstablecimientoAndPuntoEmisionAndSecuencial(establecimiento, puntoEmision, secuencial);
}

}

And in a service that implements Quartz I invoke the 3 different types of services through which: I retrieve the information from a database, I generate a few xmls, I insert records for bitacora in the second database and if this action is correct I update a state of the records retrieved from the first base, then I make a digital signature on the generated xmls And if this action is executed correctly I make a change of state in the records of the second database and insert two tables type master and detail of the third database

Then the code with which I make the invocation:

@Service
public class ScheduleService implements Serializable {
      @Autowired
      private LocalidadService localidadService;
      @Autowired
      private CooperativaService cooperativaService;
      @Autowired
      private BoletoTasaService boletoTasaService;
      @Autowired
      private BitacoraFacturasDetalleService bitacoraFacturasDetalleService;
      @Autowired
      private InformacionTributariaService informacionTributariaService;
      @Autowired
      private ClientesService clientesService;
      @Autowired
      private FacturasService facturasService;

      @Scheduled(cron = "${schedule.cronExpresion}")
      public void start() {
           if(XMLUtil.generarXML(factura, XML_GENERADO)) {
           LOGGER.info("XML para la factura " + SECUENCIAL_DOCUMENTO + " generado correctamente");
           //code that fills a javaBean
           //Execution of service that inserts in the database # 2
           if(bitacoraFacturasDetalleService.save(bitacoraFacturaDetalle)) {
                              LOGGER.info("Factura " + SECUENCIAL_DOCUMENTO + " registrada en bitacora correctamente");
               // object retrieved from database # 1 to be changed status not to be taken into account in future
               tasa.setStatusFacturacionElectronica("P");
               if(boletoTasaService.update(tasa)) {
               //Other post-upgrade operations

           }
      }
    }
}

The case is that this code works until a certain amount of registers (approximately 700 or 800 of the database 1), after the next action of insertion or update of the different bases the code goes to "sleep" for after much Time to run again

For the tests carried out prior to the transition to production, make copies of the 3 databases which for the purposes of this scenario did not have concurrent connections of the systems and / or interfaces that interact with them.

I do not know if the cause of the "problem" is: the programming code used, the strategy of defining the transactional objects (I have read and been advised to use JTA but from what I have read this mechanism uses only a transactional thread [a service that Controls the operations on the databases]) or if this inconvenience is presented by the concurences of other applications to the tables of the different databases

Please help if there is anything wrong with the spring configuration, the definition of transactional services or if you definitely need to use JTA for this purpose.

It is possible to indicate that previously I have used this scheme where I have one or several databases from which I extract information and only a database on which I make insertions, therefore I do not have problems; On the other hand, on the three databases is written given certain circumstances

Answer 1

As for the problem described, it's hard to tell exactly what might be wrong. Nevertheless, I can give you some tips:

1. You're not using any connection pooling at all. Try replacing the BasicDataSource with a HikariCP connection pool.
2. Instead of having a long-running transaction. Try to split it in chunks. Use Spring Integration to build a pipeline instead. Each database transaction should process only a small subset of data at a time. This way, the VACUUM has a better chance to run than if you're using a very long-running transaction.