Help me optimize this MySql query

Question

I have a MySql query that take a very long time to run (about 7 seconds). The problem seems to be with the OR in this part of the query: "(tblprivateitem.userid=?userid OR tblprivateitem.userid=1)". If I skip the "OR tblprivateitem.userid=1" part it takes only 0.01 seconds. As I need that part I need to find a way to optimize this query. Any ideas?

QUERY:

SELECT 
    tbladdeditem.addeditemid,
    tblprivateitem.iitemid,
    tblprivateitem.itemid
FROM tbladdeditem 
INNER JOIN tblprivateitem 
    ON tblprivateitem.itemid=tbladdeditem.itemid 
        AND (tblprivateitem.userid=?userid OR tblprivateitem.userid=1)
WHERE tbladdeditem.userid=?userid

EXPLAIN:

id    select_type    table            type    possible_keys    key    key_len    ref                    rows    extra
1     SIMPLE         tbladdeditem     ref     userid           userid 4          const                  293     Using where
1     SIMPLE         tblprivateitem   ref     userid,itemid    itemid 4          tbladdeditem.itemid    2       Using where

TABLES:

tbladdeditem contains 1 100 000 rows:

CREATE TABLE `tbladdeditem` (
    `addeditemid` int(11) NOT NULL auto_increment,
    `itemid` int(11) default NULL,
    `userid` mediumint(9) default NULL,
    PRIMARY KEY  (`addeditemid`),
    KEY `userid` (`userid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

tblprivateitem contains 2 700 000 rows:

CREATE TABLE `tblprivateitem` (
    `privateitemid` int(11) NOT NULL auto_increment,
    `userid` mediumint(9) default '1',
    `itemid` int(10) NOT NULL,
    `iitemid` mediumint(9) default NULL,
    PRIMARY KEY  (`privateitemid`),
    KEY `userid` (`userid`),
    KEY `itemid` (`itemid`) //Changed this index to only use itemid instead
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Answer 1

I would try this instead, on your original JOIN you have an OR associated with a parameter, move that to your WHERE clause.

SELECT 
    tbladdeditem.addeditemid,
    tblprivateitem.iitemid,
    tblprivateitem.itemid
FROM tbladdeditem 
INNER JOIN tblprivateitem 
    ON tblprivateitem.itemid=tbladdeditem.itemid 
WHERE tbladdeditem.userid=?userid
    AND (tblprivateitem.userid=?userid OR tblprivateitem.userid=1)

Answer 2

Since you have the predicate condition tbladdeditem.userid=?userid in the where clause I don't think you need it in the join condition.. Try removing it from the join condition and (If you are using the Or to handle the case where the parameter is null, then use Coalesce instead of OR ) if not leave it as an Or

-- If Or is to provide default for when (?userid is null...
      SELECT  a.addeditemid, p.iitemid,  p.itemid 
      FROM tbladdeditem a 
        JOIN tblprivateitem p
          ON p.itemid=a.itemid 
      WHERE a.userid=?userid 
         AND p.userid=Coalesce(?userid, 1)
-- if not then
      SELECT  a.addeditemid, p.iitemid,  p.itemid 
      FROM tbladdeditem a 
        JOIN tblprivateitem p
          ON p.itemid=a.itemid 
      WHERE a.userid=?userid 
         AND (p.userid=?userid Or p.userid = 1)

Second, if there is not an index on the userId column in these two tables, consider adding one.

Finally, if these all fail, try converting to two separate queries and unioning them together:

      Select  a.addeditemid, p.iitemid,  p.itemid 
      From tbladdeditem a 
        Join tblprivateitem p
          On p.itemid=a.itemid 
            And p.userId = a.Userid
      Where p.userid=?userid 
      Union
      Select a.addeditemid, p.iitemid,  p.itemid 
      From tbladdeditem a 
        Join tblprivateitem p
          On p.itemid=a.itemid 
            And p.userId = a.Userid
      Where p.userid = 1

Answer 3

UPDATE

I made my queries and schema match your original question exactly, multi-column key and all. The only possible difference is that I populated each table with two million entries. My query (your query) runs in 0.15 seconds.

delimiter $$
set @userid = 6
$$
SELECT 
    tbladdeditem.addeditemid,    tblprivateitem.iitemid,    tblprivateitem.itemid
FROM tbladdeditem 
INNER JOIN tblprivateitem 
    ON tblprivateitem.itemid=tbladdeditem.itemid 
        AND (tblprivateitem.userid=@userid or tblprivateitem.userid = 1)
WHERE tbladdeditem.userid=@userid

I have the same explain that you do, and with my data, my query return over a thousand matches without any issue at all. Being completely at a loss, as you really shouldn't be having these issues -- is it possible you are running a very limiting version of MySQL? Are you running 64-bit? Plenty of memory?

I had made the assumption that your query wasn't performing well, and when mine was, assumed I had fixed you problem. So now I eat crow. I'll post some of the avenues I went down. But I'm telling you, your query the way you posted it originally works just fine. I can only imagine your MySQL thrashing to the hard drive or something. Sorry I couldn't be more help.

PREVIOUS RESPONSE (Which is also an update)

I broke down and recreated your problem in my own database. After trying independent indexes on userid and on itemid I was unable to get the query below a few seconds, so I set up very specific multi-column keys as directed by the query. Notice on tbladdeditem the multi-column query begins with itemid while on the tblprivateitem the columns are reversed:

Here is the schema I used:

CREATE TABLE `tbladdeditem` (
  `addeditemid` int(11) NOT NULL AUTO_INCREMENT,
  `itemid` int(11) NOT NULL,
  `userid` mediumint(9) NOT NULL,
  PRIMARY KEY (`addeditemid`),
  KEY `userid` (`userid`),
  KEY `i_and_u` (`itemid`,`userid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `tblprivateitem` (
  `privateitemid` int(11) NOT NULL AUTO_INCREMENT,
  `userid` mediumint(9) NOT NULL DEFAULT '1',
  `itemid` int(10) NOT NULL,
  `iitemid` mediumint(9) NOT NULL,
  PRIMARY KEY (`privateitemid`),
  KEY `userid` (`userid`),
  KEY `itemid` (`itemid`),
  KEY `u_and_i` (`userid`,`itemid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

I filled each table with 2 million entries of random data. I made some assumptions:

• userid varies from 1 to 2000
• itemid varies between 1 and 10000

This gives each user about a thousand entries in each table.

Here are two versions of the query (I'm using workbench for my editor):

Version 1 - do all the filtering on the join.

Result: 0.016 seconds to return 1297 rows

delimiter $$
set @userid = 3
$$
SELECT 
    a.addeditemid,
    p.iitemid,
    p.itemid
FROM tblprivateitem as p
INNER JOIN tbladdeditem as a
    ON (p.userid in (1, @userid))
        AND p.itemid = a.itemid 
        AND a.userid = @userid
$$

Here's the explain:

EXPLAIN: 
id select_type table type  key     ref  rows extra
1  SIMPLE      p     range u_and_i      2150 Using where; Using index
1  SIMPLE      a     ref   i_and_u      1    Using where; Using index

Version 2 - filter up front

Result: 0.015 seconds to return 1297 rows

delimiter $$
set @userid = 3
$$
SELECT 
    a.addeditemid,
    p.iitemid,
    p.itemid
from 
  (select userid, itemid, iitemid from tblprivateitem 
      where userid in (1, @userid)) as p
  join tbladdeditem as a on p.userid = a.userid and a.itemid = p.itemid;
where a.userid = @userid
$$

Here's the explain:

id select_type table      type  key     ref               rows extra
1  PRIMARY     <derived2> ALL   null    null              2152
1  PRIMARY     a          ref   i_and_u p.itemid,const    1    Using where; Using index
2  DERIVED     p1         range u_and_i                   2150 Using where