MySQL/MariaDB InnoDB Simultaneous Transactions & Locking Behaviour

Question

As part of the persistence process in one of my models an md5 check_sum of the entire record is generated and stored with the record. The md5 check_sum contains a flattened representation of the entire record including all EAV attributes etc. This makes preventing absolute duplicates very easy and efficient.

I am not using a unique index on this check_sum for a specific reason, I want this all to be silent, ie if a user submits a duplicate then the app just silently ignores it and returns the already existing record. This ensures backwards compatibility with legacy app's and api's.

I am using Laravel's eloquent. So once a record has been created and before committing the application does the following:

            $taxonRecords = TaxonRecord::where('check_sum', $taxonRecord->check_sum)->get();

            if ($taxonRecords->count() > 0) {
                DB::rollBack();
                return $taxonRecords->first();
            }

However recently I encountered a 60,000/1 shot incident(odds based on record counts at that time). A single duplicate ended up in the database with the same check_sum. When I reviewed the logs I noticed that the creation time was identical down to the second. Further investigation of Apache logs showed a valid POST but the POST was duplicated. I presume the users browser malfunctioned or something but both POSTS arrived simultaneously resulting in two simultaneous transactions.

My question is how can I ensure that a transaction and its contained SELECT for the previous check_sum is Atomic & Isolated. Based upon my reading the answer lies in https://dev.mysql.com/doc/refman/8.0/en/innodb-locking-reads.html and isolation levels.

If transaction A and transaction B arrive at the server at the same time then they should not run side by side but should wait for the first to complete.

Answer 1

You created a classic race condition . Both transactions are calculating the checksum while they're both in progress, not yet committed. Neither can read the other's data, since they're uncommitted. So they calculate that they're the only one with the same checksum, and they both go through and commit.

To solve this, you need to run such transactions serially, to be sure that there aren't other concurrent transactions submitting the same data.

You may have to use use GET_LOCK() before starting your transaction to calculate the checksum. Then RELEASE_LOCK() after you commit. That will make sure other concurrent requests wait for your data to be committed, so they will see it when they try to calculate their checksum.