Use of LIMIT in Recursive Common Table Expression is not allowed in MySQL

Question

My goal is to construct a tree using newest MySQL's WITH RECURSIVE method.

My table is called categories which has 2 rows. The ID and the parentID row .

My Categories Table:

 . ID . | ParentID   
--------|----------
 . 1  . | null  
 . 2  . |  1
 . 3  . |  1  
 . 4  . |  1
 . 6  . |  1
 . 7  . |  1
 . 8  . |  1
 . 9  . |  1
 . 10 . |  1
 . 11 . |  13
 . 12 . |  14
 . 13 . |  12     
 .... . | ...

IDs from 2 to 9, have the same parent which is parent with ID = 1. This is what I am attempting to Limit by providing a "LIMIT 5" in the second SELECT query of my Recursive Common Table Expression.

An optical representation of the above table in a tree would be something like the following: My problem is to limit the number of children of the same level (marked as Item Y in below illustration).

+ Item X .............. (level 1)       
  + Item Y .............. (level 2)  
  + Item Y .............. (level 2)   
  + Item Y .............. (level 2) 
  + .... LIMIT to 5 Items 
+ Item X
    + Item X
      + Item X
         + Item X
             + Item X  
+ Item X

This is my mySQL Recursive Common Table Expression Query with the LIMIT clause causing the problem:

WITH RECURSIVE cte AS
(
  SELECT ID, 0 AS depth, CAST(ID AS CHAR(200)) AS path
    FROM categories WHERE parentID = 1
  UNION ALL
  SELECT c.ID, cte.depth+1, CONCAT(cte.path, ',', c.ID)
    FROM categories c 
    JOIN cte ON cte.ID = c.parentID
    WHERE FIND_IN_SET(c.ID, cte.path)=0 AND depth <= 10
    LIMIT 5
)

 SELECT * FROM cte

Logically I was expecting to sort my problem by using a LIMIT clause in the second Select part of the CTE to constrain the number of rows returned by the second SELECT statement. But it gives me an error:

This version of MySQL doesn't yet support 'ORDER BY / LIMIT over UNION in recursive Common Table Expression'

Note that I am using MySQL version 8.0 +. I understand the error is clear. But what about if I have 1 million children below the same parent? It will freeze the system !

I will greatly appreciate a workaround.

Thank you.

Answer 1

If I followed you correctly, row_number() can do what you want. The idea is to enumerate the categories rows in the recusive part, then filter on the top 5:

with recursive cte as (
    select id, 0 as depth, cast(id as char(200)) as path
    from categories 
    where parentid = 1
    union all
    select c.id, cte.depth+1, concat(cte.path, ',', c.id)
    from cte
    inner join (
        select c.*, row_number() over(partition by c.parentid order by c.id) rn
        from categories c 
    ) c on cte.id = c.parentid
    where find_in_set(c.id, cte.path) = 0 and depth <= 10 and c.rn <= 5
)
select * from cte

You can optimize this a little by pre-filtering the dataset:

with recursive 
    cats as (
        select *
        from (
            select c.*, row_number() over(partition by parentid order by id) rn
            from categories c 
        ) t
        where rn <= 5
    ),
    cte as (
        select id, 0 as depth, cast(id as char(200)) as path
        from cats 
        where parentid = 1
        union all
        select c.id, cte.depth+1, concat(cte.path, ',', c.id)
        from cte
        inner join cats c on cte.id = c.parentid
        where find_in_set(c.id, cte.path) = 0 and depth <= 10 and c.rn <= 5
    )
select * from cte