What's the proper way to UNION a list of queries in SQLAlchemy (Python)?

Question

So I have a list of SQLAlchemy Query objects and I want to UNION all of them.

For example if I have in my list the equivalent of:

SELECT id, name FROM person
SELECT id, name FROM employee

I would get the equivalent of:

(SELECT id, name FROM person)
UNION
(SELECT id, name FROM employee)

I don't know what's in the list and the list could contain many Query objects but of course the result signature of each SQLAlchemy Query object in my list is 100% identical.

At the moment I do the UNION like this:

if not q_list:
    return []

big_q = q_list.pop(0)
if q_list:
    big_q = big_q.union(*q_list)
result = [dict(row) for row in dbsession.execute(big_q)]

But for some weird reason I don't get the same result as if I were running all the queries individually then concatenating the results together. I must do the union incorrectly. How do I do the UNION then?

Answer 1

I think you might be expecting UNION ALL .

From the postgresql documentation , the second part..

UNION effectively appends the result of query2 to the result of query1 (although there is no guarantee that this is the order in which the rows are actually returned).

Furthermore, it eliminates duplicate rows from its result, in the same way as DISTINCT, unless UNION ALL is used.

Try this:

if q_list:
    big_q = big_q.union_all(*q_list)

There is also a function to perform union all , like:

q = union_all(*q_list)

Answer 2

My problem had nothing to do with the union call. My queries list to UNION was created with a function which was doing something like this on each object:

sql.expression.bindparam(
    "current_timestamp", value=start_timestamp
)

It seems that since all my query object in my list had the same params names ( current_timestamp ) they all used the same value instead of their individual value when bindparam was used.

I just had to use the unique parameter to fix my problem:

sql.expression.bindparam(
    "current_timestamp", unique=True, value=start_timestamp
)

That way they all get their start_timestamp value as expected.