对于重复查询，JDBC for mysql的性能非常慢

Question

I have a program need large amount of queries written in Java, JDBC is chosen to manipulate mysql db. The frame of my code is as follows:

PreparedStatement stmt = conn.preparedStatement(
            "SELECT name FROM users WHERE id = ?" )
Iterator<String, Double> it = map.entrySet().iterator(); 
//map is relativelly large, holding about 100,000 records, 
//each need to query mysql, and do some computing
//I record the time consuming of query & computing
while ( it.hasNext() ) {
    String id = it.next().getKey();
    stmt.setString(1, id);     //set the missing param with id
    long queryStart = System.currentTimeMillis();
    ResultSet rs = st.executeQuery();
    long queryEnd = System.currentTimeMillis();
    while ( rs.next() ) {
        //computing
    }
    rs.close();
    st.clearParameters();
    long computeEnd = System.currentTimeMillis();
    System.out.println(index+" has done...");
    System.out.println("  query time: "+ (queryEnd-queryStart));
    System.out.println("  compute time: "+ (computeEnd-queryEnd));
}

The performance is well at beginning for about 100-200 loops. But it suddenly goes down after.

The results print in console window is :

1 has done...
  query time:   0
  compute time: 0
2 has done...
  query time:   0
  compute time: 0
3 has done...
  ...
  ...
191 has done...
  query time:   1
  compute time: 0
192 has done...
  query time:   0
  compute time: 1
193 has done...
  query time: 1018
  compute time: 0
194 has done...
  query time: 1142
  compute time: 0
195 has done...
  query time: 1122
  compute time: 0

My database is at localhost. Why does this happen, what can impact the performance such dramatically?

How can I improve the performance?

BTW: the Object Statement, Connection ... is defined in java.sql, I'm not using the com.mysql.jdbc version, I don't know what's the deffernce.

Answer 1

Few tips which can help improve the performance:

1. Can you try enabling the query caching in MYSQL Server configuration? For the same please refer to : Query Caching in mysql

2. Are indexes set for this table?

Answer 2

when you issue a query , everytime it goes over network and got executed by db and come back to you. In order to reduce this roundtrip, you can execute the select query with bunch of ids.

how about writing a query like below

select name from users where id in ( id0 , id1, .... idn)

write a method to construct "id" clause and set it in the statement and execute it.

if number of ids are high , execute the same query with set of ids batch by batch.