将没有重复（不同）的不同列加入/联合到一个新表中

Question

trying since several hours but were not able to find a working solution. I am using MariaDB. I have a table with several millions of rows (MySourceTable) where I want to get the unique cell values from specific columns and want copy them into a new table.

COL_A  COL_B  COL_C  COL_D  COL_E

  1      11     a      ab     a1  
  2      12     a      cd     a2  
  3      13     a      ab     a3  
  1      14     b      ab     a2  
  2      12     c      ef     a5  
  4      15     d      gh     a1

The content of the new should be like this:

COL_A   COL_B  COL_C  COL_D  COL_E

  1      11     a      ab     a1  
  2      12     b      cd     a2  
  3      13     c      ef     a3  
  4      14     d      gh     a5  
         15 

     

Yep - the columns does not have any relation anymore after joining. Need the unique rows of this specific columns in a new target table for using them afterwards with DataTables SearchPanes filter .

Edit: This is how datatables searchpanes expect the values for the filter. If I dont distinct the column values into a new unique table, searchPanes has to go on each page refresh to several hundert thousand rows, to get the values.

What I tried so far:

DROP TABLE IF EXISTS col_names;
CREATE Table col_names 
(
    /*ID MEDIUMINT NOT NULL AUTO_INCREMENT, */
    ID MEDIUMINT NOT NULL Auto_Increment, 
    COL_A TINYTEXT, 
    COL_B TINYTEXT, 
    COL_C TINYTEXT, 
    COL_D TINYTEXT, 
    COL_E TINYTEXT
    PRIMARY KEY (ID)
);

INSERT INTO col_names (COL_A)
    Select Distinct Source_A AS COL_A FROM MySourceTable;
INSERT INTO col_names (COL_B)
    Select Distinct Source_B AS COL_B FROM MySourceTable;
INSERT INTO col_names (COL_C)
    Select Distinct Source_C AS COL_C FROM MySourceTable;
INSERT INTO col_names (COL_D)
    Select Distinct Source_D AS COL_D FROM MySourceTable;
INSERT INTO col_names (COL_E)
    Select Distinct Source_E AS COL_E FROM MySourceTable;

SELECT * FROM col_names
    ORDER BY COL_A, COL_B Desc; 

Result is not that what I am expecting. I think, I have to work with Cross Apply, Join, Union etc. Tried different things, but failed so many times.

Answer 1

You can do this with row_number() , union all and aggregation:

select max(col_a), max(col_b), . . .
from ((select row_number() over (order by col_a) as seqnum,
              col_a, null as col_b, null as col_c, null as col_d, null as col_e
       from t
       group by col_a
      ) union all
      (select row_number() over (order by col_b) as seqnum,
              null as col_a, col_b, null as col_c, null as col_d, null as col_e
       from t
       group by col_b
      ) union all
      . . .
     ) x
group by seqnum;

Answer 2

Change your requirements to a normalised output; two columns - column_name and column_value. Then, each cell in your expected results becomes a row in the normalised structure. This is much more in keeping with sql, much more flexible, doesn't have strange behaviour if one column has many more values than the rest, etc, etc.

SELECT
  DISTINCT
  lookup.*
FROM
  MySourceTable
CROSS APPLY
(
  SELECT 'COL_A', MySourceTable.Source_A
  UNION ALL
  SELECT 'COL_B', MySourceTable.Source_B
  UNION ALL
  etc, etc
)
  AS lookup(column_name, column_value)

Or perhaps...

SELECT DISTINCT 'COL_A' AS column_name, Source_A AS column_value FROM MySourceTable
UNION ALL
SELECT DISTINCT 'COL_B' AS column_name, Source_B AS column_value FROM MySourceTable
UNION ALL
etc, etc