Let's say we've got an SQL database (hsqldb) and want to run a number of queries on it which do not modify the content. This takes a long time for some queries and I would like to run the queries in multiple threads. So my question is: what is the best way to implement this?
I did not find any good samples to do this so I came up with the following (which I would love to get some comments on).
First, very briefly in words: I use thread-safe collections to access the queries and to put the results in. The queries are executed in a number of worker threads. The results are processed in the main thread which checks for new results until all threads are finished.
Now the code:
Create thread-safe collections of queries and results:
ConcurrentLinkedQueue<String> queries = new ConcurrentLinkedQueue<String>()
ConcurrentLinkedQueue<ResultSet> sqlResults = new ConcurrentLinkedQueue<ResultSet>();
Create a number of threads and start them (edited):
ExecutorService executorService = Executors.newFixedThreadPool(4);
for(int i=0; i<4; i++){
executorService.execute(new QueryThread(sqlResults, queries));
}
Within the thread class QueryThread a connection is opened and queries are executed as long as there are any left:
private class QueryThread implements Runnable {
private ConcurrentLinkedQueue<ResultSet> sqlResults;
private ConcurrentLinkedQueue<String> queries;
public QueryThread(ConcurrentLinkedQueue<ResultSet> sqlResults, ConcurrentLinkedQueue<String> queries){
this.sqlResults = sqlResults;
this.queries = queries;
}
@Override
public void run(){
Connection connThr = null;
try{
try {
connThr = DriverManager.getConnection(dbModeSave, "sa", "");
connThr.setAutoCommit(false);
} catch (SQLException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
String currentQuery;
do {
currentQuery = queries.poll(); // get and remove element from remaining queries
if (currentQuery != null) { // only continue if element was found
try {
Statement stmnt = connThr.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE,ResultSet.CONCUR_UPDATABLE);
try {
ResultSet resultSet = stmnt.executeQuery(currentQuery);
sqlResults.add(resultSet);
} catch (SQLException e) {
// (Do something specific)
} finally {
stmnt.close();
}
} catch (SQLException e) {
// (Do something specific)
}
}
} while (currentQuery != null);
} finally {
if (connThr != null) {
try {
connThr.close();
} catch (SQLException e) {
// Nothing we can do?
}
}
}
}
}
From the original thread I check, if the threads are all finished and therefore all queries were processed (edited).
while (!executorService.isTerminated()) {
try {
Thread.sleep(100);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
while (!sqlResults.isEmpty()) {
ResultSet result = sqlResults.poll();
//process result and close it in the end
}
}
Java standard sulution for parallel processing is ThreadPoolExecutor. Try it.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.