Bulk insert/update using Stateless session - Hibernate

Question

I have a requirement to insert/update more than 15000 rows in 3 tables. So that's 45k total inserts.

I used Statelesssession in hibernate after reading online that it is the best for batch processing as it doesn't have a context cache.

session = sessionFactory.openStatelessSession;
for(Employee e: emplList) {
  session.insert(e);
}
transcation.commit;

But this codes takes more than an hour to complete.

Is there a way to save all the entity objects in one go? Save the entire collection rather than doing it one by one?

Edit: Is there any other framework that can offer a quick insert?

Cheers!!

Answer 1

You should read this article of Vlad Mihalcea:

How to batch INSERT and UPDATE statements with Hibernate

Answer 2

You need to make sure that you've set the hibernate property:

hibernate.jdbc.batch_size

So that Hibernate can batch these inserts, otherwise they'll be done one at a time.

Answer 3

There is no way to insert all entities in one go. Even if you could do something like session.save(emplList) internally Hibernate will save one by one.

Accordingly to Hibernate User Guide StatelessSession do not use batch feature:

The insert(), update(), and delete() operations defined by the StatelessSession interface operate directly on database rows. They cause the corresponding SQL operations to be executed immediately . They have different semantics from the save(), saveOrUpdate(), and delete() operations defined by the Session interface.

Instead use normal Session and clear the cache from time to time. Acttually, I suggest you to measure your code first and then make changes like use hibernate.jdbc.batch_size , so you can see how much any tweak had improved your load.

Try to change it like this:

session = sessionFactory.openSession();
int count = 0;
int step = 0;
int stepSize = 1_000;
long start = System.currentTimeMillis();
for(Employee e:emplList) {
    session.save(e);
    count++;
    if (step++ == stepSize) {
        long elapsed = System.currentTimeMillis() - start;
        long linesPerSecond = stepSize / elapsed * 1_000;
        StringBuilder msg = new StringBuilder();
        msg.append("Step time: ");
        msg.append(elapsed);
        msg.append(" ms Lines: ");
        msg.append(count);
        msg.append("/");
        msg.append(emplList.size());
        msg.append(" Lines/Seconds: ");
        msg.append(linesPerSecond);
        System.out.println(msg.toString());
        start = System.currentTimeMillis();
        step = 0;
        session.clear();
    }
}
transcation.commit;

About hibernate.jdbc.batch_size - you can try different values, including some very large depending on underlying database in use and network configuration. For example, I do use a value of 10,000 for a 1gbps network between app server and database server, giving me 20,000 records per second.

Change stepSize to the same value of hibernate.jdbc.batch_size .