MySQL simple select query is slow

Question

I have a large mysql-table with about 110.000.000 items

The Table Design is:

CREATE TABLE IF NOT EXISTS `tracksim` (
`tracksimID` int(11) NOT NULL AUTO_INCREMENT,
`trackID1` int(11) NOT NULL,
`trackID2` int(11) NOT NULL,
`sim` double NOT NULL,
PRIMARY KEY (`tracksimID`),
UNIQUE KEY `TrackID1` (`trackID1`,`trackID2`),
KEY `sim` (`sim`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

Now I want to query a normal query:

SELECT trackID1, trackID2 FROM `tracksim` 
WHERE sim > 0.5 AND 
      (`trackID1` = 168123 OR `trackID2`= 168123)
ORDER BY sim DESC LIMIT 0,100

The Explain statement gives me:

+----+-------------+----------+-------+---------------+------+---------+------+----------+----------+-------------+
| id | select_type | table    | type  | possible_keys | key  | key_len | ref  | rows     | filtered | Extra       |
+----+-------------+----------+-------+---------------+------+---------+------+----------+----------+-------------+
|  1 | SIMPLE      | tracksim | range | TrackID1,sim  | sim  | 8       | NULL | 19980582 |   100.00 | Using where |
+----+-------------+----------+-------+---------------+------+---------+------+----------+----------+-------------+

The query seems to be very slow(about 185 seconds), but i don't know if it is only because of the amount of items in the table. Do you have a tip how I can speedup the query or the table-lookup?

Answer 1

With 110 million records, I can't imagine there are many entries with the track ID in question. I would have indexes such as

(trackID1, sim )
(trackID2, sim )
(tracksimID, sim)

and do a PREQUERY via union and join against that result

select STRAIGHT_JOIN
      TS2.*
   from
      ( select ts.tracksimID
           from tracksim ts
           where ts.trackID1 = 168123
             and ts.sim > 0.5
        UNION
        select ts.trackSimID
           from tracksim ts
           where ts.trackid2 = 168123
             and ts.sim > 0.5 
      ) PreQuery
      JOIN TrackSim TS2
         on PreQuery.TrackSimID = TS2.TrackSimID
   order by
      TS2.SIM DESC
   LIMIT 0, 100

Answer 2

Mostly I agree with Drap, but the following variation of the query might be even more efficient, especially for larger LIMIT:

SELECT TS2.*
FROM (
  SELECT tracksimID, sim
  FROM tracksim
  WHERE trackID1 = 168123
    AND sim > 0.5
  UNION
  SELECT trackSimID, sim
  FROM tracksim
  WHERE trackid2 = 168123
    AND ts.sim > 0.5
  ORDER BY sim DESC
  LIMIT 0, 100
) as PreQuery
JOIN TrackSim TS2 USING (TrackSimID);

Requires (trackID1, sim) and (trackID2, sim) indexes.

Answer 3

Try filtering your query so you don't return the full table. Alternatively you could try applying an index to the table on one of the track ID's, for example:

CREATE INDEX TRACK_INDEX
ON tracksim (trackID1)

http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html

http://www.tutorialspoint.com/mysql/mysql-indexes.htm