I have a Spring boot application with application.yml
:
spring:
datasource:
driver-class-name: org.postgresql.Driver
url: *my_url*
password: *my_pass*
username: *my_username*
jpa:
properties:
hibernate:
jdbc:
batch_size: 15
#order_inserts: true
#order_updates: true
#batch_versioned_data: true
When I try to save 200 000 entities using the method saveAll(Iterable<S> entities)
, it saves all 200 000 entities at the same time, but I want to save batches of 15 entities at a time.
Is it possible to use Spring Data's SimpleJpaRepository
and Hibernate's batch?
My approach ))
@Service
public class BulkService {
@PersistenceContext
private EntityManager em;
// @Value("${spring.jpa.properties.hibernate.jdbc.batch_size}")
private int batchSize = 20;
private List<Entity> savedEntities;
public Collection<Entity> bulkSave(List<Entity> entities) {
int size = entities.size();
savedEntities = new ArrayList<>(size);
try {
for (int i = 0; i < size; i += batchSize) {
int toIndex = i + (((i + batchSize) < size) ? batchSize : size - i);
processBatch(entities.subList(i, toIndex));
em.flush();
em.clear();
}
} catch (Exception ignored) {
// or do something...
}
return savedEntities;
}
@Transactional
protected void processBatch(List<Entity> batch) {
for (Entity t : batch) {
Entity result;
if (t.getId() == null) {
em.persist(t);
result = t;
} else {
result = em.merge(t);
}
savedEntities.add(result);
}
}
}
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.