如果两个流程同时修改两个事务中的数据并且在表上存在唯一约束，将会发生什么？

Question

I am thinking about a race condition in a production system I am working on. Database is PostgreSQL. Application is written in Java, but this is not relevant.

There is a table called "versions", which contains columns "entity_ID" and "version" (and some other fields). This table contains versions of a certain entity.

There is an application where user can modify those entities.

Every modification of an entity creates a new version to the tabel "versions" (using a trigger). This trigger finds the last version in the same table "versions" and inserts a new row with the same entity_ID, but version = (last version + 1).

There is a nightly job that is run in PostgreSQL every 4:00 that also changes those entities and therefore updates data in the table "versions". This procedure was designed to finish its work by the morning (before users of the application start to use it), but unfortunately runs into the day. As this procedure is run in a function, then it is one big transaction. Therefore the changes done by it are not visible to the application.

The nightly job uses the following workflow :

• Set "failed_counter" = 0
• Iterate over entities that need to be modified
• Do modifications to the entity inside a BEGIN .. EXCEPTION .. END block
• If there is an EXCEPTION, increase the "failed_counter". Log the exception and the failed entity to a log table.
• If "failed_counter" > 10, cancel work.
• End work

This has caused the following race condition to happen a few times (lets assume that X is the last version of entity A):

1. Nightly job starts
2. Nightly job modifies entity A, creating version X+1
3. Application is used to also modify entity A, creating also version X+1 (because the nightly job transaction has not COMMITed, so the version X+1 is not visible to the application)
4. Nightly job ends, causing COMMIT
5. There are now two versions with version number X+1, which causes application to break.

I thought that I could just solve the problem by using an UNIQUE CONSTRAINT over fields (entity_ID, version). I thought that it would cause the application to receive an error (due to violating the UNIQUE CONSTRAINT) at race condition step 3. But I am not sure how does the unique constraint work in this situation. In race condition step 3, when the application adds a version, does the database check the UNIQUE CONSTRAINT? I suppose not, since the transaction of the nightly process has not been completed. If I am correct and the UNIQUE CONSTRAINT is checked only at race condition step 4, when COMMIT is made, then this causes the whole nightly procedure to fail, which is not desired result.

So, the question is the following.

• When is the UNIQUE CONSTRAINT checked: At race condition step 3 or race condition step 4?
• If the answer to the last question is "race condition 4", then how could I change the design of the system to avoid the above-mentioned problems?

Answer 1

By default, unique constraints in PostgreSQL are checked at the end of each statement. It's easy to test the behavior using psql.

Some big, red flags . . .

As this procedure is run in a function, then it is one big transaction.

It's not one, big transaction because you're running a function. It's one, big transaction because you haven't run the function several times over smaller subsets of the data. Whether you can run the function over subsets is application-dependent.

Iterate over entities that need to be modified

Rough rule of thumb for SQL databases: iteration is always a mistake.

SQL is a set-oriented language. Dealing with sets is usually faster than iteration, and often by several orders of magnitude.

If "failed_counter" > 10, cancel work.

This looks suspicious. Why are nine failures ok? Why are any failures ok?

I thought that I could just solve the problem by using an UNIQUE CONSTRAINT over fields (entity_ID, version).

That you don't already have a unique constraint on those two columns is a big, waving red flag. Fix this first.

The fact that an application should apparently be waiting for a batch job to finish, but isn't waiting, might or might not be a system design issue. (It smells like a system design issue.)

There is a nightly job that is run in PostgreSQL every 4:00 ...

Did you think of starting at 3:00?

Test this, but not on your production server.

• Drop the trigger.
• Add a column of type timestamp with time zone .
• Set that column's default value. Most applications will use current_timestamp , but you might want clock_timestamp() instead. Docs
• Add a unique constraint on {entity_id, new timestamp column}.

Eliminating the trigger might speed things up enough for you.