MySQL - I need to join two tables in-place and return the results (without creating a temporary table)

Question

This one deserves some explanation. Imagine I have two EVENTS, A and B, and they each occur in time, for example: AAABABBABAB B.... This constitutes a STREAM.

For various reasons, we have stored these events in their own tables in MySQL, call them TA and TB. Both tables have timestamp as the primary key. (I should have used Kafka, I know...)

The problem: I need a small client to be able to query the HISTORY of the MERGED stream from start_time to end_time. I don't think it makes sense to do the join on the client (because I'd have to query TA, then query TB, then do the join, which would take up lots of time/memory/CPU). Alternatively, a server-side join, while easy, could be problematic if SEVERAL clients all request slightly different histories at slightly overlapping times... there would be a bunch of temporary tables on the server etc.

Is there a way (and it doesn't have to be super fast) to do an in-place merge without creating a temporary table on the server? For example, the pseusdo-code might look like this:

Given TA and TB, with (itA = iterator for TA) and (itB = iterator for TB), and both iterators begin at start_time:

// An in-place merge sort
while (true)
  if (itA.time >= stop_time)
    if (itB.time >= stop_time)
      break
    output(itB++)
    continue
  if (itB.time >= stop_time)
    output(itA++)
    continue
  if (itA.time < itB.time)
    output(itA++)
    continue
  output(itB++)

That code probably isn't the most efficient, but you see what I'm doing there... The 'output' function presumably sends the partial response back to the client in a buffered way. Kind of like a coroutine or something.

Note that I'm fine if the solution is either a stored procedure OR a piece of code running on the server (we're using GoLang on the server)... I just am trying to avoid creating large temporary tables on the server is all. Maybe something with cursors could be used?

Whew, I hope that makes it clear what I'm trying to do. and thanks in advance everyone.

Answer 1

This "merges" the table based on your timestamp:

( SELECT ts, ... FROM TA )
UNION ALL
( SELECT ts, ... FROM TB )
ORDER BY ts