如何有效地比较具有不同键但可能匹配值的2个对象？

Question

I have 2 objects, 1 that comes a SQL database and the other that comes from a JSON REST API. Prior to updating a row in the db with new data from the api I want to check not only if the ID already occurs in the db (which would mean an update as opposed to an insert) but also the bulk of the properties. The reason for this is that when it does update on the database side an additional "lastUpdate" datetime gets added for later processing in PowerBI and if I check only ID then the "lastUpdate" would be fired every single time an entry from the API is already in the DB, even if its properties were not actually updated. A single object (they come in arrays) of each side is as follows:

---

Cases and Notes:

Important : This section was added as per requested the (now) accepted solution by Nina Scholz. Please keep that in mind while reading.

• If a path is not entirely traversable on the API side (ie it's parent is null) then it should return Null. An example of this is the path for the DB's side callerLocationID would be callerLocation.id on the API, however when no callerLocation is set then the parent callerLocation is already null thus making the id unreachable

• If a path is entirely traversable on both sides then the values need to be compared.

• If the path cannot be traversed on the API side that should be null . For example I could then compare callerLocationID: null with callerLocation.id as the latter is null and null would be the same as null

• Once all paths are traversed and all values are compared I need to know either "yes they are all identical" ( true ) or "no they are not identical" ( false ). No need to know where they are not identical if they aren't, the entire object is send for the update.

• The paths in the DB side are basically set in stone, if any of these are null that's because they are allowed to be so in the database and that's acceptable

• All properties on the DB side that end in ID (except incidentID ) refer to a single nested id value in the API. For example callerID references caller.id , callerBranchID references callerBranch.id and operatorID references operator.id

• Some exceptions to this are:

  • impact references impact.name
  • urgency references urgency.name
  • priority references priority.name
  • duration references duration.name
  • escalationOperator references escalationOperator.id

• all the optionalField stuff from the API can be ignored

---

// Database Object
{
    incidentID: '0dc1a10f-2899-485a-b814-f72f29c9a15a',
    status: 'secondLine',
    briefDescription: 'Support niet bereikbaar',
    callDate: '2018-04-10T19:01:00.000Z',
    lastUpdate: '2018-04-18T14:02:17.000Z',
    number: 'M1804 021',
    request: '10-04-2018 21:02 Middelkoop, Paul: \nIk kan de servicedesk niet bereiken, telkens in gesprek',
    callerID: '4e723042-0037-4e05-a362-e65c620ba734',
    callerBranchID: 'f66e7804-b57a-4418-a991-997e574ead29',
    callerLocationID: null,
    externalNumber: null,
    categoryID: 'cafb0af8-e43a-4391-ac9e-a0345abbcc4f',
    subcategoryID: 'f56099a9-7c60-45e3-94b4-6555a79d4bd7',
    callTypeID: '04b678a6-791e-4662-9bc8-97573555f15e',
    entryTypeID: 'a9c486fd-a93e-565e-bfeb-17619fafe1a8',
    branchID: null,
    locationID: null,
    impact: null,
    urgency: null,
    priority: null,
    duration: null,
    operatorID: 'a17aba85-13a7-4ac6-8c57-693a512b633e',
    operatorGroupID: 'a17aba85-13a7-4ac6-8c57-693a512b633e',
    supplierID: null,
    targetDate: '2019-04-10T15:30:00.000Z',
    onHold: false,
    onHoldDate: null,
    onHoldDuration: 0,
    feedbackMessage: null,
    feedbackRating: null,
    processingStatus: 'Afgemeld',
    completed: true,
    completedDate: '2018-04-10T19:09:00.000Z',
    closed: true,
    closedDate: null,
    closureCode: null,
    creatorID: '226082ea-8d74-4dee-ae1e-74c33c883792',
    creationDate: '2018-04-10T19:02:34.000Z',
    timeSpent: 0,
    timeSpentFirstLine: 0,
    timeSpentSecondLineAndPartials: 0,
    costs: 0,
    escalationStatus: null,
    escalationReason: null,
    escalationOperator: null,
    modifier: '226082ea-8d74-4dee-ae1e-74c33c883792',
    modificationDate: '2018-04-10T19:12:08.000Z',
    expectedTimeSpent: 0,
    majorCall: false,
    majorCallID: null,
    publishToSSD: false,
    monitored: false,
    archivingReason: null
}

and

// API Object
{
    id: '0dc1a10f-2899-485a-b814-f72f29c9a15a',
    status: 'secondLine',
    number: 'M1804 021',
    request: '10-04-2018 21:02 Middelkoop, Paul: \nIk kan de servicedesk niet bereiken, telkens in gesprek',
    requests: '/tas/api/incidents/id/0dc1a10f-2899-485a-b814-f72f29c9a15a/requests',
    action: '/tas/api/incidents/id/0dc1a10f-2899-485a-b814-f72f29c9a15a/actions',
    attachments: '/tas/api/incidents/id/0dc1a10f-2899-485a-b814-f72f29c9a15a/attachments',
    caller: {
        id: '4e723042-0037-4e05-a362-e65c620ba734',
        dynamicName: 'Mafficioli del Castelletto, Richard',
        branch: {
            id: 'f66e7804-b57a-4418-a991-997e574ead29',
            name: 'Ask Roger! Delft',
            clientReferenceNumber: '',
            timeZone: 'Europe/Amsterdam',
            extraA: null,
            extraB: null
        }
    },
    callerBranch: {
        id: 'f66e7804-b57a-4418-a991-997e574ead29',
        name: 'Ask Roger! Delft',
        clientReferenceNumber: '',
        timeZone: 'Europe/Amsterdam',
        extraA: null,
        extraB: null
    },
    callerLocation: null,
    branchExtraFieldA: null,
    branchExtraFieldB: null,
    briefDescription: 'Support niet bereikbaar',
    externalNumber: '',
    category: {
        id: 'cafb0af8-e43a-4391-ac9e-a0345abbcc4f',
        name: 'Communicatie'
    },
    subcategory: {
        id: 'f56099a9-7c60-45e3-94b4-6555a79d4bd7',
        name: 'Vaste telefonie'
    },
    callType: {
        id: '04b678a6-791e-4662-9bc8-97573555f15e',
        name: 'Klacht'
    },
    entryType: {
        id: 'a9c486fd-a93e-565e-bfeb-17619fafe1a8',
        name: 'Mondeling'
    },
    object: null,
    branch: null,
    location: null,
    impact: null,
    urgency: null,
    priority: null,
    duration: null,
    targetDate: '2019-04-10T15:30:00.000+0000',
    onHold: false,
    onHoldDate: null,
    onHoldDuration: 0,
    feedbackMessage: null,
    feedbackRating: null,
    operator: {
        id: 'a17aba85-13a7-4ac6-8c57-693a512b633e',
        status: 'operatorGroup',
        name: 'Systeembeheer'
    },
    operatorGroup: {
        id: 'a17aba85-13a7-4ac6-8c57-693a512b633e',
        name: 'Systeembeheer'
    },
    supplier: null,
    processingStatus: {
        id: '70b2967d-e248-4ff9-a632-ec044410d5a6',
        name: 'Afgemeld'
    },
    completed: true,
    completedDate: '2018-04-10T19:09:00.000+0000',
    closed: true,
    closedDate: '2018-04-10T19:12:00.000+0000',
    closureCode: null,
    timeSpent: 0,
    timeSpentFirstLine: 0,
    timeSpentSecondLineAndPartials: 0,
    costs: 0,
    escalationStatus: null,
    escalationReason: null,
    escalationOperator: null,
    callDate: '2018-04-10T19:01:00.000+0000',
    creator: {
        id: '226082ea-8d74-4dee-ae1e-74c33c883792',
        name: 'Middelkoop, Paul'
    },
    creationDate: '2018-04-10T19:02:34.000+0000',
    modifier: {
        id: '226082ea-8d74-4dee-ae1e-74c33c883792',
        name: 'Middelkoop, Paul'
    },
    modificationDate: '2018-04-10T19:12:08.000+0000',
    majorCall: false,
    majorCallObject: null,
    publishToSsd: false,
    monitored: false,
    expectedTimeSpent: 0,
    archivingReason: null,
    optionalFields1: {
        boolean1: false,
        boolean2: false,
        boolean3: false,
        boolean4: false,
        boolean5: false,
        number1: 0,
        number2: 0,
        number3: 0,
        number4: 0,
        number5: 0,
        date1: null,
        date2: null,
        date3: null,
        date4: null,
        date5: null,
        text1: '',
        text2: '',
        text3: '',
        text4: '',
        text5: '',
        memo1: null,
        memo2: null,
        memo3: null,
        memo4: null,
        memo5: null,
        searchlist1: null,
        searchlist2: null,
        searchlist3: null,
        searchlist4: null,
        searchlist5: null
    },
    optionalFields2: {
        boolean1: false,
        boolean2: false,
        boolean3: false,
        boolean4: false,
        boolean5: false,
        number1: 0,
        number2: 0,
        number3: 0,
        number4: 0,
        number5: 0,
        date1: null,
        date2: null,
        date3: null,
        date4: null,
        date5: null,
        text1: '',
        text2: '',
        text3: '',
        text4: '',
        text5: '',
        memo1: null,
        memo2: null,
        memo3: null,
        memo4: null,
        memo5: null,
        searchlist1: null,
        searchlist2: null,
        searchlist3: null,
        searchlist4: null,
        searchlist5: null
    }
}

What I have done so far already

1. Iterate over the array of API Objects and for every single one check if the ID is the array of Database Objects using Fuse.JS (with a threshold of 0 for only perfect matches)

2. Use a combination of the result from 1 and the .first, .keys and .pick methods from UnderscoreJS to determine which keys are identical between the 2 objects in the current iteration with the goal of quickly checking those

    // tdIncidents is the array of objects from the API
    // dbIncidents is the array of objects from the database
    // tdinci is my iterator, consider it the "i" in the for loop
    // at this point it has already been confirmed that both dbIncidents and tdIncidents have at least 1 entry thus using [0] won't give any problems
    const db = _.first(fuse.search(tdIncidents[tdinci].id)),
      td = tdIncidents[tdinci],
      dbKeys = _.keys(dbIncidents[0]),
      tdKeys = _.keys(tdIncidents[0]),
      identicalKeysTd = _.pick(td, (value, key) => dbKeys.includes(key)),
      identicalKeysDb = _.pick(db, (value, key) => tdKeys.includes(key));

identicalKeysTd will then result to:

{ status: 'secondLine',
  number: 'M1804 021',
  request: '10-04-2018 21:02 Middelkoop, Paul: \nIk kan de servicedesk niet bereiken, telkens in gesprek',
  briefDescription: 'Support niet bereikbaar',
  externalNumber: '',
  impact: null,
  urgency: null,
  priority: null,
  duration: null,
  targetDate: '2019-04-10T15:30:00.000+0000',
  onHold: false,
  onHoldDate: null,
  onHoldDuration: 0,
  feedbackMessage: null,
  feedbackRating: null,
  processingStatus: { id: '70b2967d-e248-4ff9-a632-ec044410d5a6', name: 'Afgemeld' },
  completed: true,
  completedDate: '2018-04-10T19:09:00.000+0000',
  closed: true,
  closedDate: '2018-04-10T19:12:00.000+0000',
  closureCode: null,
  timeSpent: 0,
  timeSpentFirstLine: 0,
  timeSpentSecondLineAndPartials: 0,
  costs: 0,
  escalationStatus: null,
  escalationReason: null,
  escalationOperator: null,
  callDate: '2018-04-10T19:01:00.000+0000',
  creationDate: '2018-04-10T19:02:34.000+0000',
  modifier:
   { id: '226082ea-8d74-4dee-ae1e-74c33c883792',
     name: 'Middelkoop, Paul' },
  modificationDate: '2018-04-10T19:12:08.000+0000',
  majorCall: false,
  monitored: false,
  expectedTimeSpent: 0,
  archivingReason: null }

identicalKeysDb will result to:

{ status: 'secondLine',
  briefDescription: 'Support niet bereikbaar',
  callDate: '2018-04-10T19:01:00.000Z',
  number: 'M1804 021',
  request: '10-04-2018 21:02 Middelkoop, Paul: \nIk kan de servicedesk niet bereiken, telkens in gesprek',
  externalNumber: null,
  impact: null,
  urgency: null,
  priority: null,
  duration: null,
  targetDate: '2019-04-10T15:30:00.000Z',
  onHold: false,
  onHoldDate: null,
  onHoldDuration: 0,
  feedbackMessage: null,
  feedbackRating: null,
  processingStatus: 'Afgemeld',
  completed: true,
  completedDate: '2018-04-10T19:09:00.000Z',
  closed: true,
  closedDate: null,
  closureCode: null,
  creationDate: '2018-04-10T19:02:34.000Z',
  timeSpent: 0,
  timeSpentFirstLine: 0,
  timeSpentSecondLineAndPartials: 0,
  costs: 0,
  escalationStatus: null,
  escalationReason: null,
  escalationOperator: null,
  modifier: '226082ea-8d74-4dee-ae1e-74c33c883792',
  modificationDate: '2018-04-10T19:12:08.000Z',
  expectedTimeSpent: 0,
  majorCall: false,
  monitored: false,
  archivingReason: null }

3. At this point I thought I could check the equality of these two identicalKeys objects using UnderscoreJS's .isEqual ( _.isEqual(identicalKeysDb, identicalKeysTd) ) but to no avail. Besides the fact that I stored some keys directly without "ID" appended in the database (this could be fixed on the DB side), the more pressing issue is that database will give null for values such as the externalNumber , but the API will give '' .

Among this latest attempt I have tried many other functions both in ES6, plain JS and UnderscoreJS (too many to mention and code that has long been deleted and no longer available on my "undo" chain) but I cannot find any efficient method at all and I really really do not want to hardcode an enormous if () to check each property against its counterpart. I do not mind requiring some node package to make this comparison easy so if that is the solution by all means please share it as well.

Those objects that are actually changed I push to an array called existingIncidents which is later returned along with any new incidents. This happens as follows:

async filterIncidents() {
    const dbIncidents = await this.getDbIncidents(this.lastFetchTimestamp),
        fuseOpts = {
            'shouldSort': true,
            'findAllMatches': true,
            'threshold': 0,
            'location': 0,
            'distance': 100,
            'maxPatternLength': 36,
            'minMatchCharLength': 36,
            'keys': ['incidentID']
        },
        fuse = new Fuse(dbIncidents, fuseOpts),
        tdIncidents = await this.getTdIncidents(this.lastFetchTimestamp);

    const existIncidents = [],
        newIncidents = [];

    if (!dbIncidents.length) {
        for (const tdinci in tdIncidents) {
            newIncidents.push(tdIncidents[tdinci]);
        }
    } else {
        for (const tdinci in tdIncidents) {
            if (fuse.search(tdIncidents[tdinci].id).length) {
                // The value checking magic I need has to happen here. Some pseudo code:
                // if (values are different) {
                existIncidents.push(tdIncidents[tdinci]);
                // } else {
                // do nothing
                // }
            } else {
                newIncidents.push(tdIncidents[tdinci]);
            }
        }
    }

    return {
        'new': newIncidents,
        'existing': existIncidents
    };
}

Edit: Added the entire function at the bottom

---

Final Edit: I'm dumping a runkit link here to the final solution implementation from Nina Schulz as I have had to adjust it to my exact use case and sharing is caring, maybe it will help someone else in the future. Permalink: https://runkit.com/favna/so-compare-objects

Answer 1

This is an approach by using an array for different styled object and the relative path to the properties to compaire of each object.

The key functionality is a single function getValue which takes an object and an array of keys to the wanted property and returns a found value or of the last found value of the chain.

The other function iterates the given relation object and shows (actually) the two values for compairing and later action, like update or other wanted actions.

 function getValue(object, keys) { return keys.reduce((o, k) => o && typeof o === 'object' ? o[k] : o, object); } function compaire(objects, relations) { relations.forEach(relation => { var values = relation.map((keys, i) => getValue(objects[i], keys)); console.log(...values); }); } var objectA = { foo: { bar: 42 }, a: { b: { c: 'baz' } }, callerLocation: null }, objectB = { fooBar: 42, nested: { abc: 'bau' }, callerLocationID: null }, objects = [objectA, objectB], relations = [ [['foo', 'bar'], ['fooBar']], [['a', 'b', 'c'], ['nested', 'abc']], [['callerLocation', 'id'], ['callerLocationID']], [['x', 'u'], ['x', 'y']] ]; compaire(objects, relations);