使性能更好地检查数据库中是否存在字符串

Question

ps = (PreparedStatement) connection.prepareStatement(
      "SELECT nm.id, nid.key, nm.name, nm.languageCode FROM odds.name as nm JOIN (odds.name_id as nid)\r\n"
    + "ON (nm.id = nid.id) where nm.name like '%' and nid.key not like \"vhc%\" and nid.key not like \"vdr%\" and nid.key not like \"vto%\" and nid.key not like \"vbl%\"\r\n"
    + "and nid.key not like \"vf%\" and nid.key not like \"vfl%\" and nid.key not like \"vsm%\" and nid.key not like \"rgs%\"\r\n"
    + "and nid.key not like \"srrgs%\" and nm.typeId=8 and nm.sourceId=-1 and nm.languageCode = 'en'");

    for(Entry <String, Tag> e : allTags.entrySet()) {
        ResultSet rs = ps.executeQuery();
        while(rs.next()) {
            if(rs.getString("name").equals(e.getValue().getTranslation(Language.EN))) {
                e.getValue().setAlternativeKey(rs.getString("name"));
                break;
            }
        }
    }
);

Do you have any Ideas how I can do this a way faster. I'll try to find a string in the database and add an extra information to my object. But I have to do this for 1265 objects, so the program runs about 80 seconds.
Thanks in advance!

Answer 1

Open the DB-client of your choice (eg HeidiSQL) and do an

explain [the select statement that is originally executed]

That way MySQL explains to you what it's doing when trying to create the result and where time gets lost.

From there you can go on eg creating indizes or changing your query to make use of existing ones.

BTW:

nm.name like '%' 

looks strange. Is that a variant of

is not null

The latter might be faster. If the texts in the other like-statements are always the same, a better performance might be achieved by checking these conditions when inserting the data, add columns of type int or boolean and save the result of this check as integer/boolean in addition to the text itself. Checking against a fixed numeric value is way faster than text searches.

Answer 2

First of all, when tackling performance problems, get yourself a profiling tool that tells you where you're spending the time, how often a given method is called and so on.

But I think the case is clear enough to give some more specific hints.

You're executing your PreparedStatement over and over again, once for every entry in allTags.entrySet() , always giving you the same results, and inside in software you filter out the lines you're interested in. So you're doing the same query 1265 times, correct?

And it's puzzling me what you're doing inside the while(rs.next()) loop. Effectively, your code does (after introducing some local variables, moving constant values out of loops, ...):

    for(Entry <String, Tag> e : allTags.entrySet()) {
        Tag tag = e.getValue();
        String translation = tag.getTranslation(Language.EN);
        ResultSet rs = ps.executeQuery();
        while(rs.next()) {
            if(rs.getString("name").equals(translation)) {
                tag.setAlternativeKey(translation);
                break;
            }
        }
    }

So, the only role of the query result seems to be to decide whether the alternative key should be set (if the translation of your Tag shows up as name in the ResultSet ) - the value is already fixed by the result of the method call getTranslation(Language.EN) , independent of any database result.

I'd suggest to do one execution of your query, collecting the name values in a HashSet names , and after that do the allTags loop setting the translation if the translation is contained in your names set. That should give the same result as your code, and probably much faster.