简体   繁体   中英

How do I prevent two Node processes from inserting the exact same database record when running simultaneously?

I have a Lambda that gets thousands of events sent to it at one time. Concurrency is left at the default, which means AWS will spin up several instances to handle the incoming events. The Lambda takes the data and inserts some data into a database if that data doesn't already exist. The Lambda is written in Node.js and uses Knex to connect to a Postgres database.

The Lambda essentially contains this logic:

Does a record with ID X exist?     
a. Yes: do nothing
b. No: create a new record with ID X.

The problem is that when 50 Lambdas spin up at the same time, they'll enter a race condition where, say, 3 or 4 of them will check for the existing record at the same time (or within microseconds of each other) and not find it, therefore inserting multiple, duplicate records.

I know one way to solve this would be to create a unique constraint on the table to prevent multiple records with ID X. Then my logic would look like this:

Does a record with ID X exist? 
a. Yes: do nothing 
b. No: create a new record with ID X.
   b.1. Did that succeed?
      a. Yes: continue on.
      b. No, it threw a unique constraint error: go back to line 1.

This seems a bit contrived, but should work. Is there a better option?

EDIT:

Here is the actual code:

let location = await Location.query().where({ external_id }).first();
if(!location){
    location = await Location.query().insert({
        name,
        external_id
    });
}

Code like this:

Does a record with ID X exist?      
a. Yes: do nothing 
b. No: create a new record with ID X.

without locking the database somehow is a race condition. Between querying for record X and creating it, some other request can create it too. Don't do it this way, ever. This is racy.

You have to look at the specific tools your database offers, but a common way to execute the above sequence is to set up the database so that it doesn't allow duplicates for ID X and then you just attempt to create the record with ID x. Then, it will atomically either get created or return an error and there will be no opportunity for a race condition. You just look for the error and handle it

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM