Get a multiple UNION ALL query faster

Question

I have an issue of performance with a query with multiple UNION ALL statements. I need to add (row by row) data from different tables into the same columns. The query need to be used to create a view in MySQL, so, here an example:

CREATE OR REPLACE
    ALGORITHM = UNDEFINED 
    DEFINER = usr 
    SQL SECURITY DEFINER
VIEW my_view AS
SELECT DISTINCT 
   column 1, 
   column 2, 
   column 3
FROM 
   table 1
WHERE 
   condition 1
UNION ALL
SELECT DISTINCT 
   column 1, 
   column 2, 
   column 3
FROM 
   table 2
WHERE 
   condition 2
UNION ALL
SELECT DISTINCT 
   column 1, 
   column 2, 
   column 3
FROM 
   table 3
WHERE 
   condition 3

It seems pointless to do all the multiple UNION ALLs just to add (row by row) data from the same features (not just 3 columns as in the example, I have many more) coming from different tables because this is something that requires lots of resources from the DB, leading to "lost connection error during the query" due to the time it takes to run.

Is there any way to optimize this kind of query?

Thanks in advance.

Answer 1

UNION ALL is the most performant way of concatenating result sets. ( UNION is slower because it removes duplicates.)

Surely your timeout occurs when you use the view, not when you create it.

Your performance issue stems from one or more of the SELECT queries in your UNION ALL cascade being very slow. You, or your "data engineer" colleagues, may need to create appropriate indexes on your table 1 , table 2 , table 3 tables.

To figure this out, do these things.

• Read Optimizing Queries With EXPLAIN .
• Run SHOW CREATE TABLE whateverTableName; . Look at the output. It will show you the indexes.
• Run the SELECT queries using that same table prefixed with EXPLAIN . It will show you the indexes it used to satisfy the query.
• Ask another question here showing us the output from those two steps.

Or, it's possible your resultset from your big query is vast. There's no magic that can process millions of rows faster than O(n).