简体   繁体   中英

How can I do massive insert using Hibernate?

I am pretty new to Spring Data JPA and Hibernate and I have the following problem with massive record insertion.

I have the following situation:

1) A RoomMediaDAO Spring Data JPA interface:

@Repository
@Transactional(propagation = Propagation.MANDATORY)
public interface RoomMediaDAO extends JpaRepository<RoomMedia, Long> {

   ....................................................................
   ....................................................................
   ....................................................................
}

2) Into another class I use this RoomMediaDAO to persist a list of RoomMedia object. The list is pretty big and heavy because contains many object and each oof these objects contains a byte[] field representing a BLOB on the mapped table.

So I have the follosing statment that persist the List mediaListToBeInserted :

roomMediaDAO.save(mediaListToBeInserted);

It work fine but it is very slow because it perform the insert one by one, infact in the console I can see something like this:

Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
............................................................................
............................................................................
............................................................................

So this is not a good solution.

How can I say to Hibernate to insert more record with a single insert statment? I mean can I do something like this using Hibernate?

INSERT INTO MyTable ( Column1, Column2 ) VALUES
( Value1, Value2 ), 
( Value1, Value2 ),
( Value1, Value2 ),
.................................
.................................
.................................
( Value1, Value2 )

I absolutly need something like this to improve the performance of my batch.

This link provides a detailed explanation about different ways of performing batch inserts. Mentioned in Italics are some problems with these approaches.

  1. Using batch_size (13.1 section in the link provided) - In case you've alternating inserts (non-sequential inserts into a table, Table1 followed by Table2 and again Table1 and soon) then remember to use order_inserts .
    Also, doesn't work if Generator is of type Identity .
  2. Using StatelessSession for insertion. (13.3) - Inserted values are not stored in First Level Cache.
    May result in data aliasing effects (due to missing FLC), Also bypasses Hibernate's event model and interceptors .
  3. Using DML-style operations
    ( Insert with Select is Only supported, not values )
  4. If you're using Postgres (Or for other DB, check if those DB's support any Bulk insert Operation , then leveraging on that is a very good option). Making use of Copy manager features to upload all this data in one shot. Comparison of Performance .
    Make Sure the PushbackReader used is not overflown. I misjudged this and had to face some prod. issues with BufferOverflow.

PS : Each of the approaches have their adv & dis-adv. Choose the one that suits you.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM