Variance Analysis with SQL and Tableau

Question

Here is the situation. Right now we have two databases. Database A is used by my department. We input data into various tables (web based front end) on a quarterly basis. Once that activity is complete, all that data is pushed into corresponding tables on Database B (which is a repository for the entire business). I'm not involved with pushing this data from Database A to Database B, and I'm a little fuzzy how so much variance is happening in the first place, however, I have been tasked with improving the variance analysis activities that happen afterwards. Some IT person somewhere at some point set up a procedure that runs and dumps data in a variance analysis table (which is then extracted into an excel sheet) and is set up like this:

CRAPPY TABLE

ID A.Col A B.Col A Flag Col A A.Col B B.Col B Flag Col B 1 Yellow Yellow 0 5 5 0 2 Yellow Green 1 3 2 1 3 Blue Blue 0 7 7 0 4 Red Blue 1 4 2 1 5 Yellow Red 1 1 3 1

One of the tables we are trying to perform variance analysis on has 40 columns, and the resulting variance analysis table has 120+ columns in it: the column in database a, the column in database b, and the flag column that indicates if there is a variance between the two columns (0 if there is no variance, 1 if there is a variance).

As you can imagine, importing this extract into Tableau from the Excel file is very limiting in terms of what visualizations I can pull off. I have access to the backend of both database A and database B, and I can actually connect Tableau directly to the tables in their respective databases (rather than relying on the excel extract or the table the excel extract is coming from) and use Custom SQL to pull the data in how I want, and am hoping there is a way to use SQL to achieve a different table layout, more like this:

BETTER TABLE

ID Flag Name Value A Value B 2 Flag A Yellow Green 2 Flag B 3 2 4 Flag A Red Blue 4 Flag B 4 2 5 Flag A Yellow Red 5 Flag B 1 3

Ideally, as above, this new layout would be able to exclude any rows/columns where there isn't a variance, but if that can't be achieved, I would be happy to have an additional column called 'Flag Value' and just filter out the zero's.

So, how would I write a select statement in SQL to turn this:

Database A

ID Col A Col B 1 Yellow 5 2 Yellow 3 3 Blue 7 4 Red 4 5 Yellow 1

Database B

ID Col A Col B 1 Yellow 5 2 Green 2 3 Blue 7 4 Blue 2 5 Red 3

Into the BETTER TABLE so I can actually use this data in a meaningful way in Tableau? Thank you mucho for your help, Stack Overflow SQL experts!

Answer 1

My approach would be to use a nested query containing a union. You can probably achieve this using pivot/unpivot, although I often find that syntax cumbersome, especially when the tables have many columns.

Instead try something like:

MS SQL Server

SELECT a.ID, [Flag Name], [Value A], [Value B]
FROM (SELECT a.ID
          , [Flag Name] = CASE WHEN a.ColA != b.ColA THEN 'Flag A' ELSE NULL END
          , a.ColA [Value A]
          , b.ColA [Value B]
    FROM TableA a
    JOIN TableB b on a.ID = b.ID
    UNION ALL 
    SELECT b.ID
         , [Flag Name] = CASE WHEN a.ColB != b.ColB THEN 'Flag B' ELSE NULL END
         , CAST(a.ColB AS VARCHAR(1)) [Value A]
         , CAST(b.ColB AS Varchar(1)) [Value B]
    FROM TableA a
    JOIN TableB b on a.ID = b.ID
) a 
WHERE [Value A] != [Value B]
ORDER BY ID

MySQL

SELECT a.ID, `Flag Name`, `Value A`, `Value B`
FROM (SELECT a.ID
          , CASE WHEN a.ColA != b.ColA THEN 'Flag A' ELSE NULL END AS`Flag Name`
          , a.ColA `Value A`
          , b.ColA `Value B`
    FROM TableA a
    JOIN TableB b on a.ID = b.ID
    UNION ALL 
    SELECT b.ID
         , CASE WHEN a.ColB != b.ColB THEN 'Flag B' ELSE NULL END AS `Flag Name` 
         , CAST(a.ColB AS CHAR(1)) `Value A`
         , CAST(b.ColB AS CHAR(1)) `Value B`
    FROM TableA a
    JOIN TableB b on a.ID = b.ID
) a 
WHERE `Value A` != `Value B`
ORDER BY ID