How to update without using flush:true

Question

I have to update a 250.000 row table contaning streets. As long as I use

street.save(flush:true) 

it works, but it is terribly slow ( 2 hours for 250.000 ). If I Omit the flush:true clause it seems to work fast, but does no update at all.

Is there a need for an overall commiting, flushing after the 250.000 saves are done and how to perform it ?

peter

Answer 1

See this . It should help you all the way. Below is the code much relevant to you.

        List  batch =[]
        (0..250000).each{
           YourDomain domain= new YourDomain(....)
            batch.add(domain)
            if(batch.size()>1000){
                YourDomain.withTransaction{
                    for(YourDomain domain in batch){
                        domain.save()
                    }
                }
                batch.clear()
            }
          session = sessionFactory.getCurrentSession()
          session.clear()             
        }

Another shorter solution: You may flush records not each time but may say after 500 records. flushing each record basically takes time due to steps involved by hibernate . We could reduce these by reducing number of calls to DB.

your_record_list.eachWithIndex{ street, i ->
  street.doSomeCalc()
  street.save flush:0 == i % 500
}

Answer 2

Using domainInstance.save(flush: true) forces the SQL query to execute at that moment. This doesn't mean the changes are committed to the DB at that moment, but nevertheless it does execute some SQL.

Using a transactional method

I think the first attempt worth trying (because it's the simplest) is to ensure the method calling save() is @Transactional . That allows Hibernate to use it's best judgement, (to delay the SQL for as long as possible) rather than having to forcibly flush the Hibernate session.

import org.springframework.transaction.annotation.Transactional

class SomeController {

    @Transactional
    def save() {
        def streets = /* Some GORM query */

        streets.each {
            /* Make updates here */
            it.save() /* No flush needed because the method is transactional */
        }
    }
}

Using updateAll()

There's another option which is available only if:

1. You don't need to update properties on associations.
2. You're not doing any other updates to the same domain instances before this update.

The second criteria is subtle and can drive you mad. DetachedCriteria.updateAll(Map) does not see uncommitted changes. For instance, if you manually make a change...

someInstance.foo = 'bar'
someInstance.save(flush: true)

...and then use updateAll() to operate on the same instance, updateAll() will not know about what you've done before hand.

So, if circumstances allow, you can use updateAll() :

@Transactional
def save() {
    def streets = Street.where { /* a GORM where query */}

    streets.updateAll([propertyA: valueA, propertyB: valueB])
}

As you can tell, updateAll() modifies each qualifying domain class instance according to the property-name/property-value Map .