Node.js: Receive chained data while saving to database asynchronously

Question

I'm using sockets in JS (Node.js) to communicate with a remote host. After I connected to the remote host, the remote host sends me an array of data. When I receive the data, I need to parse the data and store them in my database (using Sequelize). The entries are chained together. So, entry 1 must exist before I can write entry 2 to my database.

The problem is that saving to the database is asynchronous while receiving the data is sychronous. Let's say I receive the first 10 database entries in the "data"-event of the Socket, I parse the first entry and save the entry to the database. Saving to the database is asynchronous and so it's possible that while saving the first entry to the database, the "data"-event happens again (receiving entry 11 till 20). So I parse again and save entry 11 to my database. But when saving entry 11, the previous 10 aren't saved...

How to solve this issue? I already thought about the blukCreate()-function but this doesn't solve the problem because I need any "event" or "point" where I can say something like "wait for async operation before receiving next data". Any idea how to solve this problem?

Answer 1

You could use a non-blocking lock:

   const createLock = () => {   
     let chain = Promise.resolve();
     return (task) => {
       const result = chain.then(() => task());
       chain = result.catch(() => {});
       return result;
     };
  };

That way you can use the lock like this:

  const useDB = createLock();

  const addEntry = entry => useDB(async () => {
    await db.add(entry); 
  });

   addEntry(0);
   addEntry(1); // will only be added after 0 is done

Answer 2

saving is async but you can use promise or await to wait for db to write your data

 const transaction = await db.sequelize.transaction() try { for (let data of dataset) { await db.YourModel.create(data, {transaction}) } } catch(e) { transaction.rollback() return } transaction.commit()