Simple MySQL JSON query takes very long time to resolve

Question

A very simple query like select * from vw_Json_Data where Ts < date_sub( now(), interval 1 day ) order by Ts desc limit 1 took > 5 minutes to resolve!

Monitoring the query using mysqladmin processlist showed that it was spending all that time 'Creating sort index'. How to reduce the query time something sensible (< 1 sec)?

The view vw_Json_Data is created as follows:

create table My_Json_Data (
  Id int auto_increment primary key,
  Tag_Id int, -- foreign key to the Tag table
  Ts datetime,
  Json_Data json
);

create table Tag (
  Tag_Id int auto_increment primary key,
  Val int
);

create or replace view vw_Json_Data as select * from Tag inner join My_Json_Data using( Tag_Id );

Currently, the Tag table contains about 2 million rows and the My_Json_Data contains 500,000 rows. Therefore, this should not be a problem for MySQL at all, albeit the Json_Data in My_Json_Data are quite complex and long JSON strings.

Answer 1

After much experimentation, what can be inferred is this. The extremely long 'Creating sort index' phase is due to the presence of JSON data. What it seems to be doing is creating sort indexes for the complex JSON for all rows that may match the query, instead of finding the row(s) that actually match first and then creating index for the JSON data from these found rows. Hence, the more rows in the table the much worse it will get.

The following, which is clearly more cumbersome but does the same thing as the original query, returns in 0.8 secs:

select *
from (
  -- grab the My_Json_Data.Id but DON'T grab the JSON data
  select A.*, Id
  from Tag A
  inner join My_Json_Data B using( Tag_Id )
  where Ts < date_sub( now(), interval 1 day )
  order by Ts desc
  limit 1
) A
inner join B My_Json_Data using( Tag_Id, Id );

If there's a better solution, please let me know:).