PDO much slower than mysql query?

Question

I am using a PDO object in PHP to run MYSQL queries, and I have been trying to work on a query to speed it up. The query is as follows, as seen in my PHP file:

$query = 
"SELECT SQL_NO_CACHE f.position, s.item_id, s.item_type, s.title, s.caption, s.date
FROM apiv2.search_all s 
INNER JOIN apiv2.tags t 
USING(item_id, item_type) 
LEFT JOIN apiv2.featured f 
ON t.item_id = f.item_id AND t.item_type = f.item_type AND f.feature_type = :id 
WHERE t.tag = 'FeaturedContent'
ORDER BY position IS NULL, position ASC, date";

$mysql_vars[':id'] = $id;
$stmt = $connection->prepare($query);
$stmt->execute($vars);
$data = $stmt->fetchAll(PDO::FETCH_ASSOC);

This query runs significantly differently depending on whether I include the ORDER BY clause, but only at the MYSQL console:

 - MYSQL Console with ORDER BY: 1.07 sec
    - starting  0.000141
    - Opening tables    0.001416
    - System lock   0.000003
    - Table lock    0.000007
    - init  0.000021
    - checking permissions  0.000001
    - checking permissions  0.000036
    - optimizing    0.000027
    - statistics    0.000027
    - preparing 0.000015
    - Creating tmp table    0.003440
    - executing 0.000001
    - Copying to tmp table  1.050029
    - Sorting result    0.013097
    - Sending data  0.000295
    - end   0.000002
    - removing tmp table    0.002045
    - end   0.000003
    - query end 0.000002
    - freeing items 0.000331
    - logging slow query    0.000002
    - cleaning up   0.000007
 - MYSQL Console unordered output: .0053 sec
    - starting  0.000076
    - Opening tables    0.001506
    - System lock   0.000002
    - Table lock    0.000005
    - init  0.000015
    - checking permissions  0.000001
    - checking permissions  0.000022
    - optimizing    0.000012
    - statistics    0.000021
    - preparing 0.000011
    - executing 0.000001
    - Sending data  0.002895
    - end   0.000002
    - query end 0.000001
    - freeing items 0.000078
    - logging slow query    0.000000
    - cleaning up   0.000002
 - PDO Prepared stmt ORDER BY: 1.18 sec
    - 'Status' => 'starting', 'Duration' => '0.000147'
    - 'Status' => 'Opening tables', 'Duration' => '0.000784'
    - 'Status' => 'System lock', 'Duration' => '0.000004'
    - 'Status' => 'Table lock', 'Duration' => '0.000007'
    - 'Status' => 'init', 'Duration' => '0.000017'
    - 'Status' => 'checking permissions', 'Duration' => '0.000002'
    - 'Status' => 'checking permissions', 'Duration' => '0.000028'
    - 'Status' => 'optimizing', 'Duration' => '0.000018'
    - 'Status' => 'statistics', 'Duration' => '0.000026'
    - 'Status' => 'preparing', 'Duration' => '0.000014'
    - 'Status' => 'Creating tmp table', 'Duration' => '0.002112'
    - 'Status' => 'executing', 'Duration' => '0.000001'
    - 'Status' => 'Copying to tmp table', 'Duration' => '1.033056'
    - 'Status' => 'Sorting result', 'Duration' => '0.013759'
    - 'Status' => 'Sending data', 'Duration' => '0.073144'
    - 'Status' => 'end', 'Duration' => '0.000003'
    - 'Status' => 'removing tmp table', 'Duration' => '0.001999'
    - 'Status' => 'end', 'Duration' => '0.000004'
    - 'Status' => 'query end', 'Duration' => '0.000007'
    - 'Status' => 'freeing items', 'Duration' => '0.000118'
    - 'Status' => 'logging slow query', 'Duration' => '0.000001'
    - 'Status' => 'cleaning up', 'Duration' => '0.000003'
 - PDO Prepared stmt unordered output: 1.06 sec
    - 'Status' => 'starting', 'Duration' => '0.000074'
    - 'Status' => 'Opening tables', 'Duration' => '0.001364'
    - 'Status' => 'System lock', 'Duration' => '0.000004'
    - 'Status' => 'Table lock', 'Duration' => '0.000007'
    - 'Status' => 'init', 'Duration' => '0.000017'
    - 'Status' => 'checking permissions', 'Duration' => '0.000001'
    - 'Status' => 'checking permissions', 'Duration' => '0.000026'
    - 'Status' => 'optimizing', 'Duration' => '0.000015'
    - 'Status' => 'statistics', 'Duration' => '0.000027'
    - 'Status' => 'preparing', 'Duration' => '0.000013'
    - 'Status' => 'executing', 'Duration' => '0.000002'
    - 'Status' => 'Sending data', 'Duration' => '1.048216'
    - 'Status' => 'end', 'Duration' => '0.000003'
    - 'Status' => 'query end', 'Duration' => '0.000001'
    - 'Status' => 'freeing items', 'Duration' => '0.000122'
    - 'Status' => 'logging slow query', 'Duration' => '0.000001'
    - 'Status' => 'cleaning up', 'Duration' => '0.000003' 
 - MYSQL functions from PHP unordered: 1.09 sec
    - 'Status' => 'starting', 'Duration' => '0.000109'
    - 'Status' => 'checking permissions', 'Duration' => '0.000002'
    - 'Status' => 'checking permissions', 'Duration' => '0.000001'
    - 'Status' => 'checking permissions', 'Duration' => '0.000004'
    - 'Status' => 'Opening tables', 'Duration' => '0.002101'
    - 'Status' => 'System lock', 'Duration' => '0.000004'
    - 'Status' => 'Table lock', 'Duration' => '0.000009'
    - 'Status' => 'init', 'Duration' => '0.000032'
    - 'Status' => 'checking permissions', 'Duration' => '0.000003'
    - 'Status' => 'checking permissions', 'Duration' => '0.000057'
    - 'Status' => 'optimizing', 'Duration' => '0.000033'
    - 'Status' => 'statistics', 'Duration' => '0.000065'
    - 'Status' => 'preparing', 'Duration' => '0.000032'
    - 'Status' => 'executing', 'Duration' => '0.000005'
    - 'Status' => 'Sending data', 'Duration' => '1.000079'
    - 'Status' => 'end', 'Duration' => '0.000005'
    - 'Status' => 'query end', 'Duration' => '0.000002'
    - 'Status' => 'freeing items', 'Duration' => '0.000300'
    - 'Status' => 'logging slow query', 'Duration' => '0.000001'
    - 'Status' => 'cleaning up', 'Duration' => '0.000007'
 - MYSQL functions from PHP w/ ORDER BY: 1.09 sec
    - 'Status' => 'starting', 'Duration' => '0.000148'
    - 'Status' => 'checking permissions', 'Duration' => '0.000001'
    - 'Status' => 'checking permissions', 'Duration' => '0.000001'
    - 'Status' => 'checking permissions', 'Duration' => '0.000001'
    - 'Status' => 'Opening tables', 'Duration' => '0.000559'
    - 'Status' => 'System lock', 'Duration' => '0.000002'
    - 'Status' => 'Table lock', 'Duration' => '0.000006'
    - 'Status' => 'init', 'Duration' => '0.000019'
    - 'Status' => 'checking permissions', 'Duration' => '0.000001'
    - 'Status' => 'checking permissions', 'Duration' => '0.000030'
    - 'Status' => 'optimizing', 'Duration' => '0.000018'
    - 'Status' => 'statistics', 'Duration' => '0.000025'
    - 'Status' => 'preparing', 'Duration' => '0.000015'
    - 'Status' => 'Creating tmp table', 'Duration' => '0.001828'
    - 'Status' => 'executing', 'Duration' => '0.000001'
    - 'Status' => 'Copying to tmp table', 'Duration' => '0.958071'
    - 'Status' => 'Sorting result', 'Duration' => '0.013502'
    - 'Status' => 'Sending data', 'Duration' => '0.088148'
    - 'Status' => 'end', 'Duration' => '0.000003'
    - 'Status' => 'removing tmp table', 'Duration' => '0.002037'
    - 'Status' => 'end', 'Duration' => '0.000003'
    - 'Status' => 'query end', 'Duration' => '0.000001'
    - 'Status' => 'freeing items', 'Duration' => '0.000112'
    - 'Status' => 'logging slow query', 'Duration' => '0.000001'
    - 'Status' => 'cleaning up', 'Duration' => '0.000004'

EXPLAINS:

WITH ORDERING

|-id-|-select_type-|-table----|-type---|-possible_keys--------|-key-----------------|-key_len-|-ref-------------------------------------|-rows--|-Extra
|-1--|-SIMPLE------|-s--------|-ALL----|-PRIMARY,search_index-|-NULL----------------|-NULL----|-NULL------------------------------------|-92166-|-Using temporary; Using filesort
|-1--|-SIMPLE------|-tags-----|-eq_ref-|-PRIMARY--------------|-PRIMARY-------------|-426-----|-apiv2.s.item_id,apiv2.s.item_type,const-|-1-----|-Using where; Using index
|-1--|-SIMPLE------|-featured-|-ref----|-type_position_index--|-type_position_index-|-62------|-const-----------------------------------|-3-----|-Using index

WITHOUT ORDERING

|-id-|-select_type-|-table----|-type---|-possible_keys--------|-key-----------------|-key_len-|-ref-------------------------------------|-rows--|-Extra
|-1--|-SIMPLE------|-s--------|-ALL----|-PRIMARY,search_index-|-NULL----------------|-NULL----|-NULL------------------------------------|-88346-|-
|-1--|-SIMPLE------|-tags-----|-eq_ref-|-PRIMARY--------------|-PRIMARY-------------|-426-----|-apiv2.s.item_id,apiv2.s.item_type,const-|-1-----|-Using where; Using index
|-1--|-SIMPLE------|-featured-|-ref----|-type_position_index--|-type_position_index-|-62------|-const-----------------------------------|-3-----|-Using index

TABLE: search_all

CREATE TABLE `search_all` (  
  `item_id` varchar(20) NOT NULL DEFAULT '',  
  `item_type` varchar(20) NOT NULL DEFAULT '',  
  `title` varchar(255) DEFAULT NULL,  
  `caption` text,  
  `date` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,  
  PRIMARY KEY (`item_id`,`item_type`),  
  UNIQUE KEY `search_index` (`item_id`,`item_type`,`date`),  
  KEY `date_index` (`date`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  


|-id-|-select_type-|-table------|-type-|-possible_keys-|-key--|-key_len-|-ref--|-rows--|-Extra
|-1--|-SIMPLE------|-search_all-|-ALL--|-NULL----------|-NULL-|-NULL----|-NULL-|-74785-|-

TABLE: tags

CREATE TABLE `tags` (  
  `tag` varchar(100) NOT NULL DEFAULT '',  
  `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,  
  `item_id` varchar(20) NOT NULL DEFAULT '',  
  `item_type` varchar(20) NOT NULL DEFAULT '',  
  PRIMARY KEY (`item_id`,`item_type`,`tag`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  


|-id-|-select_type-|-table-|-type--|-possible_keys-|-key-----|-key_len-|-ref--|-rows---|-Extra
|-1--|-SIMPLE------|-tags--|-index-|-NULL----------|-PRIMARY-|-426-----|-NULL-|-197400-|-Using index

TABLE: featured

CREATE TABLE `featured` (  
  `position` int(10) NOT NULL DEFAULT '0',  
  `item_type` varchar(20) NOT NULL DEFAULT '',  
  `item_id` varchar(20) NOT NULL DEFAULT '',  
  `feature_type` varchar(20) NOT NULL DEFAULT '',  
  PRIMARY KEY (`position`,`item_type`,`item_id`,`feature_type`),  
  KEY `type_position_index` (`feature_type`,`position`)  
) ENGINE=InnoDB DEFAULT CHARSET=utf8;  


|-id-|-select_type-|-table----|-type--|-possible_keys-|-key-----------------|-key_len-|-ref--|-rows-|-Extra
|-1--|-SIMPLE------|-featured-|-index-|-NULL----------|-type_position_index-|-66------|-NULL-|-13---|-Using index

Why is there no difference between the unordering in the PDO or MSQL from PHP? Is there any way I could make it faster?

Answer 1

Unless your order by is for an indexed column and the optimizer thinks its a good idea to use the index an "ORDER BY" will result in a sort which takes significant time and resources for any result set of any size.

The reason the PDO version is taking longer is that your script is consuming rows slower than SQLServer can serve them up, so the Database is waiting for your program. As you are using an interpreted language and an API which does a considerable amount of manipulation of the row result this is only to be expected. This delay "sending data" is much larger (0.75 sec and 1.04 sec) than the overhead from the sort.

Note the slow retrieval is compounded by the order by. In the case of the unordered SQL the rows will be sent as soon as they are selected. In the case of the ordered select all the rows will have been selected and then sorted before the first result row is sent.

Answer 2

Can you please post the tables definitions (fields, types of fields, etc)?

I would first add a simple index on date in table search_all and

a simple index on tag in table tags .