Optimizing query in the MySQL slow-query log

Question

Our database is set up so that we have a credentials table that hold multiple different types of credentials (logins and the like). There's also a credential_pairs table that associates some of these types together (for instance, a user may have a password and security token).

In an attempt to see if a pair match, there is the following query:

SELECT DISTINCT cp.credential_id FROM credential_pairs AS cp
INNER JOIN credentials AS c1 ON (cp.primary_credential_id   = c1.credential_id)
INNER JOIN credentials AS c2 ON (cp.secondary_credential_id = c2.credential_id)
WHERE c1.data = AES_ENCRYPT('Some Value 1', 'encryption key')
AND   c2.data = AES_ENCRYPT('Some Value 2', 'encryption key');

This query works fine and gives us exactly what we need. HOWEVER, it is constantly showing in the slow query log (possibly due to lack of indexes?). When I ask MySQL to "explain" the query it gives me:

+----+-------------+-------+------+--------------------------------------------------------+---------------------+---------+-------+-------+--------------------------------+
| id | select_type | table | type | possible_keys                                          | key                 | key_len | ref   | rows  | Extra                          |
+----+-------------+-------+------+--------------------------------------------------------+---------------------+---------+-------+-------+--------------------------------+
|  1 | SIMPLE      | c1    | ref  | credential_id_UNIQUE,credential_id,ix_credentials_data | ix_credentials_data | 22      | const |     1 | Using where; Using temporary   |
|  1 | SIMPLE      | c2    | ref  | credential_id_UNIQUE,credential_id,ix_credentials_data | ix_credentials_data | 22      | const |     1 | Using where                    |
|  1 | SIMPLE      | cp    | ALL  | NULL                                                   | NULL                | NULL    | NULL  | 69197 | Using where; Using join buffer |
+----+-------------+-------+------+--------------------------------------------------------+---------------------+---------+-------+-------+--------------------------------+

I have a feeling that last entry (where it shows 69197 rows) is probably the problem, but I am FAR from a DBA... help?

---

credentials table:

CREATE TABLE  `credentials` (
  `hidden_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `credential_id` varchar(255) NOT NULL,
  `data` blob NOT NULL,
  `credential_status` varchar(100) NOT NULL,
  `insert_date` datetime NOT NULL,
  `insert_user` int(10) unsigned NOT NULL,
  `update_date` datetime DEFAULT NULL,
  `update_user` int(10) unsigned DEFAULT NULL,
  `delete_date` datetime DEFAULT NULL,
  `delete_user` int(10) unsigned DEFAULT NULL,
  `is_deleted` tinyint(1) NOT NULL DEFAULT '0',
  PRIMARY KEY (`hidden_id`,`credential_id`),
  UNIQUE KEY `credential_id_UNIQUE` (`credential_id`),
  KEY `credential_id` (`credential_id`),
  KEY `data` (`data`(10)),
  KEY `credential_status` (`credential_status`(10))
) ENGINE=InnoDB AUTO_INCREMENT=1572 DEFAULT CHARSET=utf8;

credential_pairs Table:

CREATE TABLE  `credential_pairs` (
  `hidden_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `credential_id` varchar(255) NOT NULL,
  `primary_credential_id` varchar(255) NOT NULL,
  `secondary_credential_id` varchar(255) NOT NULL,
  `is_deleted` tinyint(1) DEFAULT NULL,
  PRIMARY KEY (`hidden_id`,`credential_id`),
  KEY `primary_credential_id` (`primary_credential_id`(10)),
  KEY `secondary_credential_id` (`secondary_credential_id`(10))
) ENGINE=InnoDB AUTO_INCREMENT=500 DEFAULT CHARSET=latin1;

credentials Indexes:

+-------------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| Table       | Non_unique | Key_name             | Seq_in_index | Column_name   | Collation | Cardinality | Sub_part | Packed | Null | Index_type     | Comment |
+-------------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+
| credentials |          0 | PRIMARY              |            1 | hidden_id     | A         |      186235 |     NULL | NULL   |      | BTREE      |         |
| credentials |          0 | PRIMARY              |            2 | credential_id | A         |      186235 |     NULL | NULL   |      | BTREE      |         |
| credentials |          0 | credential_id_UNIQUE |            1 | credential_id | A         |      186235 |     NULL | NULL   |      | BTREE      |         |
| credentials |          1 | credential_id        |            1 | credential_id | A         |      186235 |     NULL | NULL   |      | BTREE      |         |
| credentials |          1 | ix_credentials_data  |            1 | data          | A         |      186235 |       20 | NULL   |      | BTREE      |         |
+-------------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+

credential_pair Indexes:

+------------------+------------+---------------------------------------------+--------------+-------------------------+-----------+-------------+----------+--------+------+------------+---------+
| Table            | Non_unique | Key_name                                    | Seq_in_index | Column_name             | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+------------------+------------+---------------------------------------------+--------------+-------------------------+-----------+-------------+----------+--------+------+------------+---------+
| credential_pairs |          0 | PRIMARY                                     |            1 | hidden_id               | A         |       69224 |     NULL | NULL   |      | BTREE      |         |
| credential_pairs |          0 | PRIMARY                                     |            2 | credential_id           | A         |       69224 |     NULL | NULL   |      | BTREE      |         |
| credential_pairs |          1 | ix_credential_pairs_credential_id           |            1 | credential_id           | A         |       69224 |       36 | NULL   |      | BTREE      |         |
| credential_pairs |          1 | ix_credential_pairs_primary_credential_id   |            1 | primary_credential_id   | A         |       69224 |       36 | NULL   |      | BTREE      |         |
| credential_pairs |          1 | ix_credential_pairs_secondary_credential_id |            1 | secondary_credential_id | A         |       69224 |       36 | NULL   |      | BTREE      |         |
+------------------+------------+---------------------------------------------+--------------+-------------------------+-----------+-------------+----------+--------+------+------------+---------+

---

UPDATE NOTES:

AFAICT: The DISTINCT was superfluous... nothing really needed it, so I dropped it. In an attempt to follow Fabrizio's advice to get a where on the credential_pairs lookup I then altered the statement to read as:

SELECT credential_id 
FROM credential_pairs cp
WHERE cp.primary_credential_id = (SELECT credential_id FROM credentials WHERE data = AES_ENCRYPT('value 1','enc_key')) AND
    cp.secondary_credential_id = (SELECT credential_id FROM credentials WHERE data = AES_ENCRYPT('value 2','enc_key'))

And.... nothing. The statement takes just as long and the explain looks pretty much the same. So, I added an index to the primary and secondary columns with:

ALTER TABLE credential_pairs ADD INDEX `idx_credential_pairs__primary_and_secondary`(`primary_credential_id`, `secondary_credential_id`);

And... nothing.

+----+-------------+-------------+-------+---------------------+---------------------------------------------+---------+------+-------+--------------------------+
| id | select_type | table       | type  | possible_keys       | key                                         | key_len | ref  | rows  | Extra                    |
+----+-------------+-------------+-------+---------------------+---------------------------------------------+---------+------+-------+--------------------------+
|  1 | PRIMARY     | cp          | index | NULL                | idx_credential_pairs__primary_and_secondary | 514     | NULL | 69217 | Using where; Using index |
|  3 | SUBQUERY    | credentials | ref   | ix_credentials_data | ix_credentials_data                         | 22      |      |     1 | Using where              |
|  2 | SUBQUERY    | credentials | ref   | ix_credentials_data | ix_credentials_data                         | 22      |      |     1 | Using where              |
+----+-------------+-------------+-------+---------------------+---------------------------------------------+---------+------+-------+--------------------------+

It says it's using the index, but it still looks like it's table scanning. So, I added a joint key (as per a'r's comment below) with:

ALTER TABLE credential_pairs ADD KEY (primary_credential_id, secondary_credential_id);

And... same result as with the index (are these functionally the same?).

Answer 1

The DISTINCT is what is generating the "Use temporary", you usually want to avoid those when possible

Plus you are scanning the whole credential_pair table as you do not have any conditions against it so no indexes are used and the whole table is returned before applying the WHERE

hope it makes sense

EDIT/ADD

Try by starting from a different table, if I understand correctly, you have Table A, a Table B and a Table AB and you are starting the select from AB, try to start it from A

I haven't tested this, but you could try:

SELECT cp.credential_id
FROM credentials AS c1
    LEFT JOIN credential_pairs AS cp ON (c1.credential_id = cp.primary_credential_id)
    LEFT JOIN credentials AS c2 ON (cp.secondary_credential_id = c2.credential_id)
WHERE
    c1.data = AES_ENCRYPT('Some Value 1', 'encryption key')
    AND c2.data = AES_ENCRYPT('Some Value 2', 'encryption key');

I have had luck in the past by moving select tables around