How to simplify a complex mysql full outer join

Question

I am trying to create a full outer join in Mysql. I found several answers to the basic question, and I'm using "union" to make it work. However, I was unable to get the syntax correct without resorting to creating a few temporary tables. I've tried to generate the query without the tables, but I was never able to get the results to include the entries with a null partner_id.

Here is a reduced set of the data, already filtered by meeting_id:

+-----+---------+--------+------------+------------+
| pid | first   | gender | meeting_id | partner_id |
+-----+---------+--------+------------+------------+
|   2 | Vicki   | F      |         74 |       NULL |
|  54 | Fazal   | M      |         74 |          4 |
|   4 | Lisa    | F      |         74 |         54 |
|  10 | Rod     | M      |         74 |         57 |
|  57 | Kellee  | F      |         74 |         10 |
|  11 | Jake    | M      |         74 |         55 |
|  55 | Rosa    | F      |         74 |         11 |
|  47 | Ralph   | M      |         74 |         46 |
|  46 | Holly   | F      |         74 |         47 |
|  40 | Wes     | M      |         74 |         12 |
|  12 | Lori    | F      |         74 |         40 |
|   5 | Richard | M      |         74 |          6 |
|   6 | Rita    | F      |         74 |          5 |
|  15 | John    | M      |         74 |         16 |
|  16 | Corie   | F      |         74 |         15 |
+-----+---------+--------+------------+------------+

My original query looked like this:

set @mtg=74;

select
    a.pid,
    concat(a.first, ' ', a.last) as guy,
    a.issub as guysub,
    b.pid,
    concat(b.first, ' ', b.last) as gal,
    b.issub as galsub,
    b.partner_id
from 
    scheduled_players a
    left outer join
    scheduled_players b
    on a.partner_id = b.pid
where
    a.gender = 'M' and a.meeting_id = @mtg and b.meeting_id = @mtg

union

select
    a.pid,
    concat(a.first, ' ', a.last) as guy,
    a.issub as guysub,
    b.pid,
    concat(b.first, ' ', b.last) as gal,
    b.issub as galsub,
    b.partner_id
from 
    scheduled_players a
    left outer join
    scheduled_players b
    on b.partner_id = a.pid
where
    a.gender = 'M' and a.meeting_id = @mtg and b.meeting_id = @mtg

;

That query did not return the single entry with a null partner_id. I read a number of answers on StackOverflow and it seemed as if the where clause could cause the outer join to revert to an inner join. In my case, I did not see how this could happen, but to test this, I decided to create temporary tables to contain the 'where' clause elements. I needed to create 2 temporary tables for each of the 'guys' and 'gals', since I had the tables 2 times in the query. The results are here:

set @mtg=74;

create temporary table if not exists 
meeting_guys as select * from scheduled_players
where meeting_id = @mtg and gender='M';

create temporary table if not exists 
meeting_gals as select * from scheduled_players
where meeting_id = @mtg and gender='F';

create temporary table if not exists 
meeting_guys2 as select * from scheduled_players
where meeting_id = @mtg and gender='M';

create temporary table if not exists 
meeting_gals2 as select * from scheduled_players
where meeting_id = @mtg and gender='F';


select
    a.pid,
    concat(a.first, ' ', a.last) as guy,
    a.issub as guysub,
    b.pid,
    concat(b.first, ' ', b.last) as gal,
    b.issub as galsub,
    b.partner_id
from 
    meeting_guys a
    left outer join
    meeting_gals b
    on a.partner_id = b.pid

union

select
    a.pid,
    concat(a.first, ' ', a.last) as guy,
    a.issub as guysub,
    b.pid,
    concat(b.first, ' ', b.last) as gal,
    b.issub as galsub,
    b.partner_id
from 
    meeting_guys2 a
    right outer join
    meeting_gals2 b
    on b.partner_id = a.pid
;

It turned out this worked, and I received the results I was expecting (I removed the last names since these are real people):

+------+---------+--------+------+--------+--------+------------+
| pid  | guy     | guysub | pid  | gal    | galsub | partner_id |
+------+---------+--------+------+--------+--------+------------+
|   54 | Fazal   |      0 |    4 | Lisa   |      0 |         54 |
|   10 | Rod     |      0 |   57 | Kellee |      0 |         10 |
|   11 | Jake    |      0 |   55 | Rosa   |      0 |         11 |
|   47 | Ralph   |      0 |   46 | Holly  |      0 |         47 |
|   40 | Wes     |      0 |   12 | Lori   |      0 |         40 |
|    5 | Richard |      0 |    6 | Rita   |      0 |          5 |
|   15 | John    |      0 |   16 | Corie  |      0 |         15 |
| NULL | NULL    |   NULL |    2 | Vicki  |      0 |       NULL |
+------+---------+--------+------+--------+--------+------------+

I was able to get the results I was looking for, but I don't understand why the previous query did not work. Fortunately, I have a working solution, but I'd really like to find out if there is a better, more optimal way.

Answer 1

Firstly to point out that this is untested so you might just need to tweak it but you sound more than capable of fixing the odd error. If you do need me to clarify why I did something or you need me to fix something, just say the word.

To explain why your first attempt eliminated the null records unexpectedly, you are right that it is your where clause that is doing it. for the left join, instead of a.meeting_id = @mtg and b.meeting_id = @mtg you would use `a.meeting_id = @mtg and (b.meeting_id = @mtg or b.meeting_id is null)' obviously for the right join you would check for the null in the left table.

As for an alternate solution, I have used a temp table to limit the result set to just the matching meeting_id's early (for performance) in case your table is large, and then I filter for M/F in the derived tables.

Hope it helps you.

set @mtg=74;

create temporary table if not exists 
meeting as 
select
    pid,
    concat(first, ' ', last) as full_name,
    issub,
    partner_id,
    meeting_id,
    gender
from scheduled_players
where meeting_id = @mtg;

select
    M.pid,
    M.full_name as guy,
    M.issub as guysub,
    F.pid,
    F.full_name as gal,
    F.issub as galsub,
    F.partner_id
from 
    (select * from meeting where gender = 'M') M
    left outer join (select * from meeting where gender = 'F') F
        on M.partner_id = F.pid
UNION
select
    M.pid,
    M.full_name as guy,
    M.issub as guysub,
    F.pid,
    F.full_name as gal,
    F.issub as galsub,
    F.partner_id
from 
    (select * from meeting where gender = 'M') M
    right outer join (select * from meeting where gender = 'F') F
        on F.partner_id = M.pid

EDIT

If performance isn't an issue then maybe it is just simpler to forget the temp table altogether and refer to the table directly in the derived tables as;

select concat(first, ' ', last) as full_name, * from scheduled_players where gender = 'M' and meeting_id = @mtg
select concat(first, ' ', last) as full_name, * from scheduled_players where gender = 'F' and meeting_id = @mtg

You could also create a single temp table and then insert and update that in separate queries.

Whatever works for you at the end of the day.