Get the intersection of two many-to-many relationship of specific values

Question

NB I have tagged this with SQLAlchemy and Python because the whole point of the question was to develop a query to translate into SQLAlchemy. This is clear in the answer I have posted. It is also applicable to MySQL.

I have three interlinked tables I use to describe a book. (In the below table descriptions I have eliminated extraneous rows to the question at hand.)

MariaDB [icc]> describe edition;
+-----------+------------+------+-----+---------+----------------+
| Field     | Type       | Null | Key | Default | Extra          |
+-----------+------------+------+-----+---------+----------------+
| id        | int(11)    | NO   | PRI | NULL    | auto_increment |
+-----------+------------+------+-----+---------+----------------+
7 rows in set (0.001 sec)

MariaDB [icc]> describe line;
+------------+--------------+------+-----+---------+----------------+
| Field      | Type         | Null | Key | Default | Extra          |
+------------+--------------+------+-----+---------+----------------+
| id         | int(11)      | NO   | PRI | NULL    | auto_increment |
| edition_id | int(11)      | YES  | MUL | NULL    |                |
| line       | varchar(200) | YES  |     | NULL    |                |
+------------+--------------+------+-----+---------+----------------+
5 rows in set (0.001 sec)

MariaDB [icc]> describe line_attribute;
+------------+------------+------+-----+---------+-------+
| Field      | Type       | Null | Key | Default | Extra |
+------------+------------+------+-----+---------+-------+
| line_id    | int(11)    | NO   | PRI | NULL    |       |
| num        | int(11)    | YES  |     | NULL    |       |
| precedence | int(11)    | YES  | MUL | NULL    |       |
| primary    | tinyint(1) | NO   | MUL | NULL    |       |
+------------+------------+------+-----+---------+-------+
5 rows in set (0.001 sec)

line_attribute.precedence is the hierarchical level of the given heading. So if War and Peace has Books > Chapters, all of the lines have an attribute that corresponds to the Book they're in (eg, Book 1 has precedence=1 and num=1 ) and an attribute for the Chapter they're in (eg, Chapter 2 has precedence=2 and num=2 ). This allows me to translate the hierarchical structure of books with volumes, books, parts, chapters, sections, or even acts and scenes. The primary column is a boolean, so that each and every line has one attribute that is primary. If it is a book heading, it is the Book attribute, if it is a chapter heading, it is the Chapter attribute. If it is a regular line in text, it is a line attribute, and the precedence is 0 since it is not a part of the hierarchical structure.

I need to be able to query for all lines with a particular edition_id and that also have the intersection of two line_attributes.

(This would allow me to get all lines from a particular edition that are in, say, Book 1 Chapter 2 of War and Peace).

I can get all lines that have Book 1 with

SELECT
    line.*
FROM
    line
INNER JOIN
    line_attribute
ON
    line_attribute.line_id=line.id
WHERE
    line.edition_id=2 AND line_attribute.precedence=1 AND line_attribute.num=1;

and I can get all lines that have Chapter 2:

SELECT
    line.*
FROM
    line
INNER JOIN
    line_attribute
ON
    line_attribute.line_id=line.id
WHERE
    line.edition_id=2 AND line_attribute.precedence=2 AND line_attribute.num=1;

Except the second query returns each chapter 2 from every book in War and Peace.

How do I get from these two queries to just the lines from book 1 chapter 2?

Answer 1

Warning from Raymond Nijland in the comments:

Note for future readers.. Because this question is tagged MySQL.. MySQL does not support INTERSECT keyword.. MariaDB is indeed a fork off the MySQL source code but supports extra features which MySQL does not support.. In MySQL you can simulate the INTERSECT keyword with a INNER JOIN or IN()

Trying to write up a question on SO helps me get my thoughts clear and eventually solve the problem before I have to ask the question. The queries above are much clearer than my initial queries and the question pretty much answers itself, but I never found a clear answer that talks about the intersect utility, so I'm posting this answer anyway.

The solution was the INTERSECT operator.

The solution is simply the intersection of those two queries:

SELECT
    line.*
FROM
    line
INNER JOIN
    line_attribute
ON
    line_attribute.line_id=line.id
WHERE
    line.edition_id=2 AND line_attribute.precedence=1 AND line_attribute.num=1

INTERSECT    /* it is literally this simple */

SELECT
    line.*
FROM
    line
INNER JOIN
    line_attribute
ON
    line_attribute.line_id=line.id
WHERE
    line.edition_id=2 AND line_attribute.precedence=2 AND line_attribute.num=2;

This also means I could get all of the book and chapter headings for a particular book by simply adding an additional constraint ( line_attribute.primary=1 ).

This solution seems broadly applicable to me. Assuming, for instance, you have questions in a StackOverflow clone, which are tagged, you can get the intersection of questions with two tags (eg, all posts that have both the SQLAlchemy and Python tags). I am certainly going to use this method for that sort of query.

I coded this up in MySQL because it helps me get the query straight to translate it into SQLAlchemy.

The SQLAlchemy query is this simple:

[nav] In [10]: q1 = Line.query.join(LineAttribute).filter(LineAttribute.precedence==1, LineAttribute.num==1)                              

[ins] In [11]: q2 = Line.query.join(LineAttribute).filter(LineAttribute.precedence==2, LineAttribute.num==1)                              

[ins] In [12]: q1.intersect(q2).all()  

Hopefully the database structure in this question helps someone down the road. I didn't want to delete the question after I solved my own problem.