SQL Query performs slower for another mysql version?

Question

Currently I'm working on a project which requires me to write an API for my users. I have written the following SQL statement so far and tested it on my localhost over my php:

SELECT topquery.kind_id,
       topquery.image,
       topquery.id,
       userquery.user_name
FROM `order` AS topquery,
     `user` AS userquery
WHERE userquery.user_id = topquery.user_id
  AND topquery.id IN
    (SELECT MIN(id)
     FROM `order` AS mainquery
     WHERE user_id != '$vUserId'
       AND id NOT IN
         (SELECT order_id
          FROM history
          WHERE user_id = '$vUserId')
       AND kind_id NOT IN
         (SELECT o.kind_id
          FROM history h
          INNER JOIN `order` o ON h.order_id = o.id
          WHERE h.user_id = '$vUserId')
       AND actions > 0
       AND kind = '1'
     GROUP BY kind_id)
ORDER BY vip DESC
LIMIT 35

This query takes about 0.5 secs on my localhost, and takes ~1.4 seconds on my server.

Now my question: Is it possible that the speed of SQL queries can vary for each mysql version? Or is my query just not "optimized" enough?

Answer 1

Yes, it will be different, because every version of sql has different tweaks to be optimized or code path or algorithm processing speed. I think the latest version would faster than previous one.

Answer 2

off topic advice, but I wanted formatting so a comment isn't enough

You commence your query using a join predicate within the where clause

FROM `order` AS topquery,
     `user` AS userquery
WHERE userquery.user_id = topquery.user_id

Yet deeper into the query you do use INNER JOIN syntax. So why not do this throughout?

FROM `order` AS topquery,
INNER JOIN `user` AS userquery ON userquery.user_id = topquery.user_id

Stick to using only explicit joins. Don't allow yourself to use commas between tables in the from clause. Don't "mix and match" join syntax, this leads to maintenance problems and possible bugs.

Answer 3

Here I outline a diagnostic for your case, and then you can ascertain whether or not the speed is still actually a problem:

1. Network. Im not sure if thats time executed purely from SQL or if network time is in there as well. Your network call will incur an overhead.

2. It depends on what your server is, but if your environments are matched, it is almost always definitely because there is substantially more data in your server database than your localhost.

One of the plusses of having a shared dev server in a corporate context is that someone will probably put some data on there for you every so often, so everyone doesnt have to copy everything down to their machine to work with a real size data set. More data = more work = takes longer, simple as that.

3. You have a discrepancy in your database schema. If you arent in the position to rebuild your database in one command, you might've added an index on your localhost and forgot about it. You can use a schema tool to get the differences between your localhost and your database. I like SQLYog. I think Mysql workbench does it as well.

4. You have a discrepancy in your hardware. Ie your server is a 486 with 4mb of RAM.

Answer 4

Difference in performance between servers

Yes, performance can change between different MySQL versions. The reason is because the entire software might change, including the MySQL optimizer which calculates the plan that the query will be executed by.

Few factors I would check before deciding whether the MySQL version is the root cause are:

1. Hardware - do you have different hardware on your localhost and on your server? CPU, memory, hard drives, etc.
2. Allocated resources - Do you provide different memory allocations to the MySQL instance in my.cnf / my.ini configuration files on the different servers? If so, it can drastically impact the MySQL instance performance.
3. Caching - maybe the query was cached by the MySQL instance on one of the servers. I would suggest to restart the MySQL instance to make sure the caches are clear, and then run the query and compare the execution times.
4. Amount of data - do you have different amount of data in your localhost and the second server? If so, it can significantly impact the query execution time.

Query Performance

In addition, I looked at your query and it looks like it can be better optimized.

(I used EverSQL SQL Query Optimizer to get the recommendations - disclaimer, I'm a co-founder of EverSQL and humbly provide these suggestions):

1. You can replace the subselects (which are bad for performance) in the IN clause with an INNER JOIN.
2. You can replace the subselects in the NOT IN clause with a LEFT JOIN, which will improve performance.
3. Also, when you have complex sub queries, you can consider splitting them to temporary tables and join to those tables. The big advantage is that you can index the columns in that temporary table, join to it and once the session is done it will be dropped.
4. Make sure you add the relevant indexes (according to the data distribution in the columns and their cardinality). Make sure the indexes match the order of the conditions.

This is how the queries will look like after optimization:

create table temp as 
SELECT
          o.kind_id
        FROM
          history h
          INNER JOIN `order` o ON h.order_id = o.id
        WHERE
          h.user_id = '$vUserId';

CREATE INDEX kind_id on temp(kind_id);

create table temp2 as
SELECT
      MIN(id) as id
    FROM
      `order` AS mainquery
      left join history h on mainquery.id=h.order_id
      left join temp t on mainquery.kind_id=t.kind_id
    WHERE
      user_id != '$vUserId'
      AND h.order id is null
      and t.kind_id is null
      AND actions > 0
      AND kind = '1'
    GROUP BY
      kind_id;    

CREATE INDEX id on temp2(id);

SELECT 
    topquery.kind_id,
    topquery.image,
    topquery.id,
    userquery.user_name
FROM
    `order` AS topquery,
    `user` AS userquery,
    temp2 AS temp
WHERE
    userquery.user_id = topquery.user_id
        AND topquery.id = temp.id
ORDER BY vip DESC
LIMIT 35