MySQL: random effect on ANALYZE TABLE

Question

I have got 3 innodb tables, say A, B, and C. There is a query to join these three tables to generate results.

SELECT A.a, B.b, C.c
from A 
join B on A.id = B.a_id 
join C on C.id = B.c_id
where A.a = 'example' and B.b < 10;

In the beginning when I was testing the query using 'EXPLAIN' command, it gives me the following order:

B -- C -- A

However, this is not optimal. So I run 'ANALYZE TABLE' to all tables, and it gives me :

A -- B -- C

, which I believe it's the correct order.

Then I deployed the SQL to production, and for no reason, after 1 month, the execution plan switched back to the bad option, which is B--C--A. After that, I tried run ANALYZE TABLE again for several times , though, this time, the results are confusing to me. Sometimes it give me B--C--A as well, sometimes it gives me A--B--C, and sometimes even other execution plan.

So my question is:

1. why the execution plan change after deployment?
2. besides pinning the execution plan (the data get updated and change fast, so the optimal plan maybe change in the future), is there a way to guarantee that optimal plan always ensured?

Answer 1

The optimizer makes choices about reordering tables and using indexes based on in-memory statistics about the size of the table, cardinality, distribution of values, indexes etc. These statistics are estimates , not absolutely accurate at all times.

InnoDB updates its statistics from time to time, and that's what you can cause to happen when you run ANALZYE TABLE.

But still, there are cases where the stats in memory are right on the cusp of making the optimizer make a different choice, so you see this flip-flopping behavior.

You can override the optimizer's default algorithm for choosing indexes by specifying index hints in your query.

You can override the optimizer's default algorithm for reordering tables by specifying STRAIGHT_JOIN . This means you want it to read the tables in the order you gave them in your FROM clause, and don't reorder them.

You can use STRAIGHT_JOIN as a query modifier (like DISTINCT). Put it right after SELECT:

SELECT STRAIGHT_JOIN A.a, B.b, C.c
from A 
join B on A.id = B.a_id 
join C on C.id = B.c_id
where A.a = 'example' and B.b < 10;

But be careful about using index hints or join hints too liberally. The optimizer may avoid the flip-flopping behavior next week, after the size and distribution of data changes just a little bit. If you have too many overrides in your code, you could prevent the optimizer from doing a better job!