How to do a batch insert version of auto increment?

Question

I have a temporary table that has columns id and value :

| id | value |

And I have a table that aggregates data with columns g_id and value :

| g_id | value |

The id in the temporary table is locally ordered by id, here's 2 examples:

temp_table 1

| id | value  |
+----+--------+
| 1  | first  |
| 2  | second |
| 3  | third  |

temp_table 2

| id | value |
+----+-------+
| 2  | alpha |
| 3  | beta  |
| 4  | gamma |

I want to insert all the rows from the temp table into the global table, while having the g_id ordered globally. If I insert temp table 1 before temp table 2, it should be like this:

global_table

| group_id | value  |
+----------+--------+
| 1        | first  |
| 2        | second |
| 3        | third  |
| 4        | alpha  |
| 5        | beta   |
| 6        | gamma  |

For my purposes, it is ok if there are jumps between consecutive numbers, and if 2 inserts are done at the same time, it can be interleaved. The requirement is basically that the id columns are always increasing.

eg Let's say I have 2 sets of queries run at the same time, and the group_id in global_table is auto incremented:

Query 1:

INSERT INTO global_table (value) VALUES (first)
INSERT INTO global_table (value) VALUES (second)
INSERT INTO global_table (value) VALUES (third)

Query 2:

INSERT INTO global_table (value) VALUES (alpha)
INSERT INTO global_table (value) VALUES (beta)
INSERT INTO global_table (value) VALUES (gamma)

I can get something like this:

global_table

| group_id | value  |
+----------+--------+
| 1        | first  |
| 3        | alpha  |
| 4        | second |
| 5        | third  |
| 6        | beta   |
| 7        | gamma  |

How do I achieve something like this with inserting from a table? Like with

INSERT INTO global_table (value)
    SELECT t.value
    FROM temp_table t

Unfortunately, this may not result in a incrementing id all the time.

Answer 1

The requirement is basically that the id columns are always increasing.

If I understood you correctly, you should use ORDER BY in your INSERT statement.

INSERT INTO global_table (value)
SELECT t.value
FROM temp_table t
ORDER BY t.id;

In SQL Server, if you include ORDER BY t.id , it will guarantee that the new IDs generated in the global_table table will be in the specified order.

I don't know about other databases, but SQL Server IDENTITY has such guarantee.

---

Using your sample data, it is guaranteed that the group_id generated for value second will be greater than the value generated for the value first . And group_id for value third will be greater than for value second .

They may be not consecutive, but if you specify ORDER BY , their relative order will be preserved.

Same for the second table, and even if you run two INSERT statements at the same time. Generated group_id s may interleave between two tables, but relative order within each statement will be guaranteed.

---

See my answer for a similar question with more technical details and references.