Grails CSV 插件 - 并发

Question

I am using the plugin: Grails CSV Plugin in my application with Grails 2.5.3 . I need to implement the concurrency functionality with for example: GPars , but I don't know how I can do it.

Now, the configuration is sequential processing. Example of my code fragment:

Thanks.

Answer 1

Implementing concurrency in this case may not give you much of a benefit. It really depends on where the bottleneck is. For example, if the bottleneck is in reading the CSV file, then there would be little advantage because the file can only be read in sequential order. With that out of the way, here's the simplest example I could come up with:

import groovyx.gpars.GParsPool

def tokens = csvFileLoad.inputStream.toCsvReader(['separatorChar': ';', 'charset': 'UTF-8', 'skipLines': 1]).readAll()
def failedSaves = GParsPool.withPool {
    tokens.parallel
        .map { it[0].trim() }
        .filter { !Department.findByName(it) }
        .map { new Department(name: it) }
        .map { customImportService.saveRecordCSVDepartment(it) }
        .map { it ? 0 : 1 }
        .sum() 
}

if(failedSaves > 0) transactionStatus.setRollbackOnly()

As you can see, the entire file is read first; hence the main bottleneck. The majority of the processing is done concurrently with the map() , filter() , and sum() methods. At the very end, the transaction is rolled back if any of the Department s failed to save.

Note: I chose to go with a map() - sum() pair instead of using anyParallel() to avoid having to convert the parallel array produced by map() to a regular Groovy collection, perform the anyParallel() , which creates a parallel array and then converts it back to a Groovy collection.

Improvements

As I already mentioned in my example the CSV file is first read completely before the concurrent execution begins. It also attempts to save all of the Department instances, even if one failed to save. You may want that (which is what you demonstrated) or not.