我可以使用Spring Data saveAll（）方法和Hibernate 5批处理吗？

Question

I have a Spring boot application with application.yml :

spring:
    datasource:
        driver-class-name: org.postgresql.Driver
        url: *my_url*
        password: *my_pass*
        username: *my_username*
    jpa:
        properties:
            hibernate:
                jdbc:
                    batch_size: 15
                    #order_inserts: true
                    #order_updates: true
                    #batch_versioned_data: true

When I try to save 200 000 entities using the method saveAll(Iterable<S> entities) , it saves all 200 000 entities at the same time, but I want to save batches of 15 entities at a time.

Is it possible to use Spring Data's SimpleJpaRepository and Hibernate's batch?

Answer 1

My approach ))

@Service
public class BulkService {

    @PersistenceContext
    private EntityManager em;

    // @Value("${spring.jpa.properties.hibernate.jdbc.batch_size}")
    private int batchSize = 20;

    private List<Entity> savedEntities;

    public Collection<Entity> bulkSave(List<Entity> entities) {
        int size = entities.size();
        savedEntities = new ArrayList<>(size);

        try {
            for (int i = 0; i < size; i += batchSize) {
                int toIndex = i + (((i + batchSize) < size) ? batchSize : size - i);
                processBatch(entities.subList(i, toIndex));
                em.flush();
                em.clear();
            }
        } catch (Exception ignored) {
            // or do something...  
        }
        return savedEntities;
    }

    @Transactional
    protected void processBatch(List<Entity> batch) {

        for (Entity t : batch) {
            Entity result;
            if (t.getId() == null) {
                em.persist(t);
                result = t;
            } else {
                result = em.merge(t);
            }
            savedEntities.add(result);
        }
    }
}

Working example and test