Check array for identical object before pushing new object

Question

I am stumped.

I have an array of objects that looks like this:

arr = [{source:someID, target:someID},{source:someID, target:someID},...]

After a certain stage, the array's length reaches around ~20000. At this point in the running of the code, each object within the array is entirely unique.

I need to go on and add further objects to the array but I don't want any double ups. In this process, another array is looped over to create new objects when two objects share a similar key value. These new objects are then pushed to the above array. Is there a way to test so that the new objects are pushed to this array only if the above array doesn't already contain an identical object.

I'm not the most proficient in JS, and still learning. So far, I thought of using a nested loop

  let testArr = [{code: num, id:num},{code: num, id:num},{code: num, id:num},…] // object being looped over to create more objects for above arr
  let testData = testArr;
  let arr = [{{source:someID, target:someID},{source:someID, target:someID},...}] // array being added to
  let len = testArr.length;
  for (let i = 0; i < len; i++) {
    let objectTest = testData.findIndex((e) => e.uniqueID.includes(testArr[i].uniqueID));
    if (objectTest !== -1) {
      if (testArr[i].id !== testData[objectTest].id) {
        let testObj = {
           source: testArr[i].id,
           target: testData[objectTest].id,
         };
         for (let a = 0; a < arr.length; a++) {
           if (deepEqual(arr[a], testObj) === false) {
              const newLink = new Object();
              newLink.source = testArr[i].id;
              newLink.target = testData[objectTest].id;
              arr.push(newLink);
          }
        }
      }
    }
  }

For the deepEqual function I've tried numerous different iterations (most found on here) of functions designed to test if objects/arrays are identical and I don't think those functions on their own are the trouble.

When running this code, I run out of memory ( JavaScript heap out of memory ) and the app terminates. Was originally running the browser but moved it to Node. If I increased the max ram node could use to 16gb, the app would still terminate with the code: Fatal JavaScript invalid size error 156627439 .

What I'm stuck on is a valid way I can check the array to see if there is an identical object already present and then skipping over this if it is true.

Any pointers would be much appreciated.

Answer 1

The biggest issue I can see is inside this piece of code:

for (let a = 0; a < arr.length; a++) {
    if (deepEqual(arr[a], testObj) === false) {
        const newLink = new Object();
        newLink.source = testArr[i].id;
        newLink.target = testData[objectTest].id;
        arr.push(newLink);
    }
}

You are looping while a < arr.length , but you push in the same array, so the length increases. Moreover, you push an object for every entry if you don't find an equal object.
Let's say the are 10 elements and there isn't a single object as the one you want to check inside: on the first iteration deepEqual returns false and you push the element; repeat this step and you'll push 10 times the same element, and the arr.length is now 20, so there will be 10 more iterations where deepEqual returns true and you don't push a new object.

Now just image the same with 20000 elements.

You should just check if it exists in the array, THEN eventually push it. Try replacing the above with the following solution:

const doesNotContainEqual = arr.every((obj) => !deepEqual(obj, testObj));

if (doesNotContainEqual) {
    const newLink = new Object();
    newLink.source = testArr[i].id;
    newLink.target = testData[objectTest].id;
    arr.push(newLink);
}