由于高负载重新设计查询/数据库

Question

I need help with query or redesigning my db. I'm not a DB admin, nor am I some MySQL wiz.

I have the following tables:

CREATE TABLE IF NOT EXISTS `pop_contor` (
  `key` varchar(50) NOT NULL,
  `uniqueHandler` varchar(30) DEFAULT NULL,
  `uniqueLink` varchar(30) DEFAULT NULL,
  `uniqueUser` varchar(30) DEFAULT NULL,
  `owner` varchar(50) NOT NULL,
  `ip` varchar(15) DEFAULT NULL,
  `page` varchar(500) DEFAULT NULL,
  `share` float DEFAULT NULL,
  `cadv` float NOT NULL,
  `os` varchar(50) NOT NULL,
  `browsershort` varchar(50) NOT NULL,
  `browser` varchar(50) DEFAULT NULL,
  `country` varchar(10) DEFAULT NULL,
  `date` date DEFAULT NULL,
  `hour` int(2) NOT NULL,
  `tstamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  PRIMARY KEY (`key`),
  KEY `contor_IX1` (`uniqueLink`,`ip`),
  KEY `owner` (`owner`,`share`,`hour`),
  KEY `uniqueUser` (`uniqueUser`,`share`,`hour`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

and:

   CREATE TABLE IF NOT EXISTS `pop_links` (
  `unique` varchar(8) NOT NULL DEFAULT '',
  `uniqueUser` varchar(30) DEFAULT NULL,
  `uniqueCategories` varchar(150) DEFAULT NULL,
  `browser` varchar(60) NOT NULL,
  `os` varchar(50) NOT NULL,
  `country` varchar(100) NOT NULL,
  `name` varchar(50) DEFAULT NULL,
  `url` varchar(500) CHARACTER SET ascii DEFAULT NULL,
  `description` varchar(150) DEFAULT NULL,
  `bid` varchar(6) DEFAULT NULL,
  `amount` varchar(5) DEFAULT NULL,
  `remain` varchar(20) DEFAULT NULL,
  `rtoday` varchar(20) NOT NULL DEFAULT '0',
  `frequency` varchar(2) DEFAULT NULL,
  `dlimit` varchar(6) NOT NULL DEFAULT '0',
  `hours` varchar(100) NOT NULL DEFAULT 'all',
  `block` varchar(500) CHARACTER SET ascii NOT NULL,
  `valid` int(1) DEFAULT NULL,
  `payed` int(1) DEFAULT NULL,
  `startDate` date DEFAULT NULL,
  `endDate` date DEFAULT NULL,
  `date` date DEFAULT NULL,
  PRIMARY KEY (`unique`),
  KEY `unique` (`unique`,`uniqueCategories`,`browser`,`os`,`country`,`url`,`bid`,`remain`,`rtoday`,`frequency`,`hours`,`block`,`valid`,`startDate`,`endDate`,`uniqueUser`    )
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

What I'm trying to do is to show each visitor (based on ip) the best fit link based on several criteria. I've tried several queries but failed as server still gets overloaded. It hapens mostly when table pop_contor gets around 200k entries. Table links has ~30 rows in it.

First attempt was doing a select>where>select>where>select>where but it ate my lunch including my lunchbox.

Second attempt was (query took about 5 seconds to complete):

SELECT l. * FROM pop_links AS l 
LEFT JOIN ( SELECT uniqueLink, SUM( ip = '".$ip."' ) AS ip_visits 
FROM pop_contor 
GROUP BY uniqueLink ) AS c 
ON c.uniqueLink = l.unique AND ip_visits <= frequency 
WHERE (`uniqueCategories` LIKE '%,".$cat.",%'OR `uniqueCategories` = '1') 
AND (`hours` LIKE '%,".date("H").",%' OR `hours` = 'all') 
AND (`browser` LIKE '%".$user_browser[name]."%' OR `browser` = '1') 
AND (`country` LIKE '%".$ccode."%' OR `country` = '1') 
AND (`os` LIKE '%".$user_browser[platform]."%' OR `os` = '1') 
AND (`remain` > '0') 
AND (`rtoday` > '0') 
AND `valid` = '1' 
AND (`block` NOT LIKE '%".$unique."%') 
ORDER BY `bid` DESC, `remain` DESC 
LIMIT 1

Explain:

id  select_type     table   type    possible_keys   key     key_len     ref     rows    Extra
1   PRIMARY     l   ALL     NULL    NULL    NULL    NULL    16  Using where; Using temporary; Using filesort
1   PRIMARY     <derived2>  ALL     NULL    NULL    NULL    NULL    16  
2   DERIVED     pop_contor  index   NULL    contor_IX1  141     NULL    299128  Using index

Third atempt:

SELECT pop_links.unique, pop_links.uniqueUser, pop_links.uniqueCategories, pop_links.browser, pop_links.os, pop_links.country, pop_links.url, pop_links.bid, pop_links.remain, pop_links.rtoday, pop_links.frequency, pop_links.hours, pop_links.block, pop_links.valid, pop_links.startDate, pop_links.endDate, COUNT( IF( pop_contor.ip = '".$ip."', 1, NULL ) ) < pop_links.frequency AS toto 
FROM pop_links 
LEFT JOIN pop_contor 
ON pop_links.unique = pop_contor.uniqueLink 
WHERE (`uniqueCategories` LIKE '%,".$cat.",%' OR `uniqueCategories` = '1') 
AND (`hours` LIKE '%,".date("H").",%' OR `hours` = 'all') 
AND (pop_links.browser LIKE '%".$user_browser[name]."%' OR pop_links.browser = '1') 
AND (pop_links.country LIKE '%".$ccode."%' OR pop_links.country = '1')
AND (pop_links.os LIKE '%".$user_browser[platform]."%' OR pop_links.os = '1')
AND (`remain` > '0') 
AND `rtoday` > '0' 
AND `valid` = '1' 
AND (`block` NOT LIKE '%".$unique."%') 
GROUP BY pop_links.unique, pop_contor.uniqueLink 
ORDER BY `toto` DESC , pop_links.bid DESC , pop_links.remain DESC 
LIMIT 1

Explain:

id  select_type     table   type    possible_keys   key     key_len     ref     rows    Extra
1   SIMPLE  pop_links   index   NULL    unique  2683    NULL    16  Using where; Using index; Using temporary; Using f...
1   SIMPLE  pop_contor  ref     contor_IX1  contor_IX1  93  admin_pops.pop_links.unique     13277   Using index

Query completes in about 0.3 seconds but the load on the server is still high. SHOW PROCESSLIST shows me a lot of "Copying to tmp table" statuses.

Answer 1

Replaced hours LIKE '%,".date("H").",%' OR hours = 'all' with paused = 0 (added a column paused to links).

Load dropped from ~4 to ~0.20. I can't believe how much impact column LIKE %something could have on the query performance.