简体   繁体   中英

Spring data jpa batch execute all inserts operations

I need to insert a lot of data in a mysql(Something around of 100k) then I'm trying to use Spring Data Jpa to insert in batch, for that I'm using a simple example with 30 records.

The first thing was remove @GeneratedValue os my entity end implements Persistable to no need a select before a insert:

@Entity
public class User implements Persistable {

    @Id
    private Integer id;
    // properties here...

Then, on my application.yml:

spring:
  jpa:
    properties:
      hibernate.jdbc.batch_size: 30
      hibernate.generate_statistics: true
    show-sql: true
    hibernate:
      ddl-auto: validate
  datasource:
    driverClassName: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://localhost:3306/db?cachePrepStmts=true&reWriteBatchedInserts=true
    // user and password

I have a simple repository:

public interface UserRepository extends JpaRepository<User, Integer> { }

and the insert method:

public void process() {

        List<User> users = new ArrayList<>();

        for (int i = 1 ; i <= 30; i++) {
            User user = new User();
            user.setId(i);
            // set properties

            users.add(user);

            if(i % 30 == 0) {
                userRepository.saveAll(users);
                users.clear();
            }
        }
    }

Then I think the correct is 1 batch operation only, but I had 29 statements:

    1745893 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    3524622 nanoseconds spent preparing 30 JDBC statements;
    68290171 nanoseconds spent executing 29 JDBC statements;
    215125391 nanoseconds spent executing 1 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    240389888 nanoseconds spent executing 1 flushes (flushing a total of 29 entities and 29 collections);
    0 nanoseconds spent executing 0 partial-flushes (flushing a total of 0 entities and 0 collections)

Any ideias?

thanks!

Test next property:

spring.jpa.properties.hibernate.order_inserts=true

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM