Optimise query using filesort and temporary

Question

Can anyone help me to optimise this query. It's taking a while to run:

The su_pref table has almost 900,000 rows

SELECT p.mykey,
       p.merchant_name,
       m.merchant_url, 
       p.name,
       p.description,
       p.image_url, 
       p.deep_link,
       p.rrp_price, 
       p.display_price, 
       c.category,
       p.su_parent_name 
FROM su_pref p #
        INNER JOIN su_categoryrefs cr ON p.mykey = cr.mykey 
        INNER JOIN su_categories c ON cr.id = c.id 
        INNER JOIN su_merchants m ON p.merchant_id=m.id 
WHERE 
     cr.id =36 
 ORDER BY p.date_created DESC LIMIT 0,20 

And this is what I get when I use EXPLAIN

I'm really struggling to understand how to use EXPLAIN to optimise a query, so any help would be greatly appreciated.

Here are the table definitions. The su_pref table is quite big so I stripped some fields out

CREATE TABLE `su_pref` (
  `mykey` varchar(50) NOT NULL,
  `merchant_id` smallint(3) default NULL,
  `merchant_name` varchar(50) NOT NULL default '',
  `brand` varchar(50) default NULL,
  `merchantproductkey` varchar(50) default NULL,
  `upc` varchar(15) default NULL,
  `name` varchar(255) NOT NULL default '',
  `description` varchar(2500) NOT NULL default '',
  `short_description` varchar(500) default NULL,
  `thumb_url` varchar(500) default NULL,
  `image_url` varchar(500) default NULL,
  `deep_link` varchar(1000) default NULL,
  `merchant_link` varchar(255) default NULL,
  `rrp_price` decimal(11,2) default NULL,
  `display_price` decimal(11,2) default NULL,
  `delivery_cost` decimal(11,2) default NULL,
  `price_range` tinyint(1) default NULL,
  `keywords` varchar(500) default NULL,
  `su_parent_name` enum('Women','Men','') NOT NULL,
  `date_created` date default NULL,
  `date_modified` datetime default NULL,
  `wp_featured` varchar(1) default NULL,
  `hp_featured` varchar(1) default NULL,
  `published` varchar(1) default NULL,
  `in_todays_feed` varchar(1) default NULL,
  PRIMARY KEY  (`mykey`),
  KEY `merchant_id` (`merchant_id`),
  KEY `date_created` (`date_created`),
  FULLTEXT KEY `product_search` 
                (`name`,`description`,`short_description`,
                 `keywords`, `product_type`,`colour`,`style`,
                 `material`,`datafeed_category_name`,
                 `datafeed_subcategory_name`,
                 `brand`,`merchant_name`),
  FULLTEXT KEY `name` (`name`,`datafeed_category_name`, 
                       `datafeed_subcategory_name`,`product_type`,`keywords`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

-

CREATE TABLE `su_categories` (
  `id` mediumint(9) NOT NULL,
  `category` varchar(100) NOT NULL,
  `parent_id` tinyint(4) NOT NULL,
  `update_query` varchar(3000) default NULL,
  `url` varchar(150) default NULL,
  `last_update` datetime default NULL,
  PRIMARY KEY  (`id`,`category`,`parent_id`),
  KEY `parent_id` (`parent_id`),
  KEY `category_id` (`id`),
  FULLTEXT KEY `category_name` (`category`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

-

CREATE TABLE `su_categoryrefs` (
  `mykey` varchar(255) NOT NULL,
  `id` smallint(4) NOT NULL,
  PRIMARY KEY  (`mykey`,`id`),
  KEY `id` (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1

-

CREATE TABLE `su_merchants` (
  `id` mediumint(9) NOT NULL,
  `merchant_name` varchar(40) NOT NULL,
  `merchant_url` varchar(40) default NULL,
  `merchant_website` varchar(40) default NULL,
  `merchant_description` varchar(2000) default NULL,
  `merchant_featured` varchar(1) default NULL,
  PRIMARY KEY  (`id`),
  UNIQUE KEY `merchant_name` (`merchant_name`)
) ENGINE=MyISAM AUTO_INCREMENT=128 DEFAULT CHARSET=latin1

Thank you in advance

Answer 1

It's impossible to tell which table each of those unqualified columns is being returned from, or which table the "merchant_id" in the JOIN predicate is being referenced from, or which table date_created in the ORDER BY clause is being referenced from. (We can make some guesses, but they are only that, guesses.

The first step, really, is to qualify all of those column references with either the table_name, or preferably a table alias.

Also, formatting the statement so that mere mortals can comprehend it helps a great deal.

Your reformatted statement with qualified column references helps, and the inclusion of the schema definition also helps. (I will get my answered updated.)

SELECT p.mykey
     , p.merchant_name
     , merchant_url
     , `name`
     , description
     , image_url
     , deep_link
     , rrp_price
     , display_price
     , category
     , su_parent_name
 FROM su_pref p
 JOIN su_categoryrefs cr ON cr.mykey = p.mykey
 JOIN su_categories c    ON c.id     = cr.id
 JOIN su_merchants m     ON m.id     = p.merchant_id
WHERE cr.id = 36
ORDER BY date_created DESC LIMIT 0,20

As a general rule of thumb, its desirable to have an index with leading columns the columns referenced in the join predicates. Typically, these are the PRIMARY KEY and all the foreign keys.

In addition, if the are "covering indexes" (an index that contains all of the columns referenced in the query), you will see "Using index" in the EXPLAIN output; this means the query is being satisfied from the index without referencing the data blocks.

The EXPLAIN seems to show a "colour_id" index being referenced, but it's not at all obvious which columns are contained in that index. (The EXPLAIN PLAN output image is too hard for me to read.)

---

here are some suggestions:

ensure you have an indexes

ON su_categories (id, category)     -- you do
ON su_merchants (id, merchant_url)  -- never mind this one, it's a small table
ON su_categoryrefs (id, mykey)

I'm assuming that the id column in the su_categoryrefs table is a foreign key referencing su_categories.id , and not the primary key of the table. (This looks like a relationship table that resolves a many-to-many relationship between su_categories and 'su_preferences`, but I'm just guessing.