Use JSON_TABLE cross join on subquery mysql

Question

I am at a loss on how to use a JSON_TABLE in mysql 8.0 only on a small part of another table, if the selection of this part involves a JOIN or a subquery:

Let's start with a table predictions which contains the value of a predictions as well as some foreign key

SELECT * FROM predictions

id | ml_model_id | value
+--+-------------+------ 
1  | 1           | [{"class": "dog", "confidence": 1}, {"class": "cat", "confidence": 0.03}]

Here is the selection of the whole table that works well:

SELECT * 
    FROM predictions,
         JSON_TABLE(
             predictions.value,
              "$[*]" 
              COLUMNS (class VARCHAR(30) PATH "$.class",
                       confidence FLOAT PATH "$.confidence")
            ) myjson

Original problem: a subquery cannot be used as a table to build the JSON_TABLE on

However, this does not work anymore if I replace the predictions table by a subquery of it

SELECT * FROM (
    SELECT * FROM predictions
    WHERE ml_model_id = 1
    # Do any type of filtering here, or even none at all
    ) preds,
    JSON_TABLE(
        preds.value,
        "$[*]"
        COLUMNS (class VARCHAR(30) PATH "$.class",
                       confidence FLOAT PATH "$.confidence")
            ) myjson
        )

I will get an SQL Error 1210 "Incorrect argument to JSON_TABLE"

Limited workaround: If the filtering is on a table column, you can use WHERE...

My first thought was to take the filtering outside of the subquery, which to a certain extent work:

SELECT * FROM predictions, 
    JSON_TABLE(...) myjson
WHERE predictions.ml_model_id = 1

This solution works, already comes off as unelegant to me, because it means that the , between predictions and JSON_TABLE is an implicit lateral join instead of a cross join (Edit: note that even when using a CROSS JOIN instead of the , does an implicit lateral join)

...But it does not work if your filtering involves JOINs

However, if instead of a WHERE i'd want to filter by using a JOIN, I will get an even more cryptic error:

SELECT * FROM predictions, 
    JSON_TABLE(...) myjson
   INNER JOIN ml_models
       ON ml_models.id = predictions.ml_model_id
       AND ml_models.usage = "classification"

will raise an SQL ERROR [1054] Unknown column 'predictions.ml_model_id' in 'on clause' ... Why would this column not be found? Running without the inner join actually returns it !

Unelegant workaround that seems to work in all cases: Using EXISTS in WHERE clause

Simply replacing the INNER JOIN by a WHERE EXISTS works, but this is really counterintuitive to work that way.

SELECT * FROM predictions, 
    JSON_TABLE(...) myjson
   WHERE EXISTS (SELECT 1 FROM ml_models
       WHERE ml_models.id = predictions.ml_model_id
       AND ml_models.usage = "classification")

Is there something that I don't understand on JSON_TABLE that prevents me to use a subquery on it?

Answer 1

I finally managed to make it work with a subquery as first part of the join, by casting the JSON field as a JSON again in the JSON_TABLE

SELECT * FROM (
    SELECT * FROM predictions
    WHERE ml_model_id = 1
    # Do any type of filtering here, or even none at all
    ) preds,
    JSON_TABLE(
        CAST(preds.value AS JSON),
        "$[*]"
        COLUMNS (class VARCHAR(30) PATH "$.class",
                       confidence FLOAT PATH "$.confidence")
            ) myjson
        )

I am however even more at a loss on why this would work, so I will keep the question open if someone has an explanation about why it works this way and not without the cast.

Answer 2

This works for me, it appears to be an order of operations when it comes to the joins.

SELECT * FROM predictions preds
   INNER JOIN ml_models
       ON ml_models.id = preds.ml_model_id
       AND ml_models.usage = "classification", 
    JSON_TABLE(preds.value, ...) myjson