Promises & Recursion

Question

Here is the situation. I have inherited a DB schema that is difficult to work with (we have all been there before) and Node is highlighting that even more now. What I am needing to do is recursively go through the relationships of a record (one2many most of the time) and stop when the record's object equals one set in an array constant, but only for that one relationship chain. All of the other relationships need to continue on till they reach the same point as well. The amount of relationships can vary for each of our deployments, so creating a static solution is not possible.

gatherDataToSync: async (connection, recordId) => {
    self.recordIdsUsed.push(recordId);

    let objectId = await referenceHelper.getObjectIdForRecordId(connection, recordId);
    let objectData = await self.getRelatedObjectsInformation(connection, recordId);

    self.objectCollection.push({
        "recordId": recordId,
        "objectId": objectId,
        objectData
    });

    if (referenceHelper.includes(self.objectListToStopExecutionAt, objectId) === false) {
        let relationships = objectData['relationships'];

        if (relationships != null) {
            for (let i = 0; i < relationships.length; i++) {

                if (referenceHelper.includes(self.relationshipsToIgnore, relationships[i]['id']) === false) {
                    let relationshipRecordIds = relationships[i]['relationships'];

                    if (relationshipRecordIds != null) {
                        for (let j = 0; j < relationshipRecordIds.length; j++) {
                            if (referenceHelper.includes(self.recordIdsUsed, relationshipRecordIds[j]) === false) {
                                await self.gatherDataToSync(connection, relationshipRecordIds[j]);
                            }
                        }
                    }
                }
            }
        }
    }
},

That is my recursive method. I need to have all of the await s in the loop to finish processing prior to the original call to the gatherDataToSync function to return its promise back to the function it was called from. Is there a way to do this or am I not thinking about this right?

The self.recordIdsUsed and self.objectCollection are both module variables that are used to keep track of the data so that it doesn't get out of sync for comparison purposes in the loops.

Please let me know if you need more code as I was not sure what all was needed to understand what is happening.

Thank you in advance for your help!

Answer 1

I need to have all of the await s in the loop to finish processing prior to the original call to the gatherDataToSync function to return its promise back to the function it was called from.

That's not how async function s work. The promise from the call is always returned immediately (and that's a good thing). The promise is however not resolved prior to all the code in the function finishing its processing, including the pauses from await s.

It's likely that your code already works as expected as-is.

Answer 2

I'm not sure if the following code will work for you (depends how you mutate values and how referenceHelper.includes is implemented based on mutated values). The thing is that you have to return something so the caller of gatherDataToSync knows when it's done.

Your code does not return anything but if it would return await self.gatherDataToSync it would execute in the slowest possible way since it's stacking all asynchronous processing.

    gatherDataToSync: async (connection, recordId) => {
      self.recordIdsUsed.push(recordId);
      //minor improvement maybe, you can do these 2 things in parallel
      let [objectId,objectData] = await Promise.all(
        referenceHelper.getObjectIdForRecordId(connection, recordId),
        self.getRelatedObjectsInformation(connection, recordId)
      );

      self.objectCollection.push({
          "recordId": recordId,
          "objectId": objectId,
          objectData
      });

      if (referenceHelper.includes(self.objectListToStopExecutionAt, objectId) === false) {
          //you return a recursive promise here so you can await calling gatherDataToSync
          return Promise.all(
            //no need for if statement and for loop, just take out all items you don't need
            (objectData['relationships'] || [])//no need for if null
            .filter(//filter out the ones you don't need
              item =>
                referenceHelper.includes(self.relationshipsToIgnore, item) !== false
            )
            .map(
              item => 
                (item['relationships'] || [])//no need for inner if null
                .filter(//filter out the ones you don't need
                  item =>
                    referenceHelper.includes(self.recordIdsUsed, item) !== false
                )
            )
            .map(
              item =>//this is an array of array of objectIds
                Promise.all(//create a promise with recursive promises
                  item.map(
                    item=>self.gatherDataToSync(connection, item)
                  )
                )
            )
        );
      }
    }