Bulkindexing JPA Entities modified during Spring transaction to Elasticsearch index

Question

I have an JPA Entity Class that is also an Elasticsearch Document. The enviroment is a Spring Boot Application using Spring Data Jpa and Spring Data Elasticsearch.

@Entity
@Document(indexname...etc)
@EntityListeners(MyJpaEntityListener.class)
public class MyEntity {
    //ID, constructor and stuff following here
}

When an instance of this Entity gets created, updated or deleted it gets reindexed to Elasticsearch. This is currently achieved with an JPA EntityListener which reacts on PostPersist, PostUpdate and PostRemove events.

public class MyJpaEntityListener {

    @PostPersist
    @PostUpdate
    public void postPersistOrUpdate(MyEntity entity) {
        //Elasticsearch indexing code gets here
    }

    @PostRemove
    public void postPersistOrUpdate(MyEntity entity) {
        //Elasticsearch indexing code gets here
    }
}

That´s all working fine at the moment when a single or a few entities get modified during a single transaction. Each modification triggers a separate index operation. But if a lot of entities get modified inside a transaction it is getting slow.

I would like to bulkindex all entities that got modified at the end (or after commit) of a transaction. I took a look at TransactionalEventListeners, AOP and TransactionSynchronizationManager but wasn´t able to come up with a good setup till now.

How can I collect all modified entities per transaction in an elegant way without doing it per hand in every service method myself?

And how can I trigger a bulkindex at the end of a transaction with the collected entities of this transaction.

Thanks for your time and help!

Answer 1

One different and in my opinion elegant approach, as you don't mix your services and entities with elasticsearch related code, is to use spring aspects with @AfterReturning in the service layer transactional methods.

The pointcut expression can be adjusted to catch all the service methods you want.

@Order(1) guaranties that this code will run after the transaction commit.

The code below is just a sample...you have to adapt it to work with your project.

@Aspect
@Component()
@Order(1)
public class StoreDataToElasticAspect {

    @Autowired
    private SampleElasticsearhRepository elasticsearchRepository;


    @AfterReturning(pointcut = "execution(* com.example.DatabaseService.bulkInsert(..))")
    public void synonymsInserted(JoinPoint joinPoint) {
        Object[] args = joinPoint.getArgs();
        //create elasticsearch documents from method params.
        //can also inject database services if more information is needed for the documents.
        List<String> ids = (List) args[0];
        //create batch from ids
       elasticsearchRepository.save(batch);
    }
}

And here is an example with a logging aspect.