How to remove duplicated OBJECTS from JavaScript array?

Question

What's the best way to remove duplicate objects from array of objects?

From

var arr = 
    [
        {"name":"Joe", "age":17}, 
        {"name":"Bob", "age":17}, 
        {"name":"Carl", "age": 35},
        {"name":"Bob", "age":35},
        {"name":"Joe", "age":17}, 
    ]

when duplicates removed, the expected result is

res= arr = 
    [
        {"name":"Joe", "age":17}, 
        {"name":"Bob", "age":17}, 
        {"name":"Carl", "age": 35},
        {"name":"Bob", "age":35},
    ]

(5 objects, 1 duplicate, 4 left).

The number of properties of each object is fixed, the properties names are the same for each array. However, from array to array they may not be just "name" and "age" as above, but the names of the properties could be any.

@Pointy Please treat the duplicate word in the question above as 'duplicate' in the verbal sense - the object with the same number of properties, the same properties and the same values of that properties respectively.

THIS IS NOT DUPLICATE OF Remove Duplicates from JavaScript Array

Answer 1

You could use an object for lookup, if an object is alreday inserted or not.

Edit:

Update for getting all properties of the object and use the values for the key. If only some properties should be used for it, then I suggest to use an array with the relavant keys, like

['name', 'age']

and use it with

var key = ['name', 'age'].map(function (k) { return a[k]; }).join('|');

 var arr = [{ "name": "Joe", "age": 17 }, { "name": "Bob", "age": 17 }, { "name": "Carl", "age": 35 }, { "name": "Bob", "age": 35 }, { "name": "Joe", "age": 17 }], filtered = arr.filter(function (a) { var key = Object.keys(a).map(function (k) { return a[k]; }).join('|'); if (!this[key]) { return this[key] = true; } }, Object.create(null)); console.log(filtered); 

Answer 2

The solution using Object.keys , Array.every and Array.concat functions:

var names = {}, result = [];
arr.forEach(function (v) {
    var name = v['name'];
    names[name] = names[name] || [];
    // considering multiple objects with same 'name' but different 'age' - and vise versa
    if (!Object.keys(names[name]).length ||  
        names[name].every((obj) => ((obj['name'] === name && obj['age'] !== v['age']) || (obj['name'] !== name && obj['age'] === v['age'])) )) {
        names[name].push(v);
    }
}, names);
Object.keys(names).forEach((k) => result = result.concat(names[k]), result);

console.log(JSON.stringify(result, 0, 4));

The output:

[
    {
        "name": "Joe",
        "age": 17
    },
    {
        "name": "Bob",
        "age": 17
    },
    {
        "name": "Bob",
        "age": 35
    },
    {
        "name": "Carl",
        "age": 35
    }
]

Answer 3

This is a general solution to filter duplicate of any flat (not nested) object from an array of objects. Not tailored to this specific question. Otherwise this job would be easily done by establishing a LUT (look up table made of a Map object for instance) according to the already known properties and values. Again.. you can apply this solution to any objects array to remove the dupes.

The best way to perform this task comes with an invention of Object.prototype.compare() So once we have it under our hand the job becomes nothing more than a game. Let's see it.

 Object.prototype.compare = function(o){ var ok = Object.keys(this); return typeof o === "object" && ok.length === Object.keys(o).length ? ok.every(k => this[k] === o[k]) : false; }; var arr = [ {"name":"Joe", "age":17}, {"name":"Bob", "age":17}, {"name":"Carl", "age": 35}, {"name":"Bob", "age":35}, {"name":"Joe", "age":17}, ], red = arr.reduce((p,c,i) => {var f = p.slice(i).findIndex(o => o.compare(p[i-1])); return f == -1 ? p : (p.splice(f+i,1),p)},arr); console.log(JSON.stringify(arr,null,2)); 

I believe this is best way of of doing this so far. I would prefer a Map or Set of course however as of this version of ES6 even though we can use objects as keys in the Map objects they still can not be used to access the values. I mean even if the key object is an empty object another empty object wouldn't define that key. But i think there is a work going on this since in Map objects NaN can be a key and you can access it with NaN even though in JS world as you all know NaN !== NaN Such as ;

var m = new Map();
m.set({},true);
m.set("a",1);
m.set(NaN,"cow");
m.get({},true};   // undefined
m.get("a",1);     // 1
m.get(NaN,"cow"); // "cow" so NaN == NaN is true in the Map ...

Answer 4

const arr1 = [{"name":"ren","age":3,"weight":120},{"name":"ren","age":2,"weight":100},{"name":"ren","age":2,"weight":100},{"name":"ren","age":2,"weight":100},{"name":"ren","age":2,"weight":100},{"name":"ren","age":2,"weight":100},{"name":"ren","age":1,"weight":100},{"name":"stimpy","age":2,"weight":100},{"name":"george american","age":56,"weight":220}]
const arr2 = [{"name":"ren","age":2,"weight":150},{"name":"ren","age":2,"weight":150},{"name":"ren","age":2,"weight":150},{"name":"ren","age":2,"weight":100},{"name":"stimpy","age":2,"weight":100},{"name":"ren","age":3,"weight":100},{"name":"stimpy","age":1,"weight":100},{"name":"ren","age":2,"weight":100},{"name":"circus midgets","age":5,"weight":200}]

function uniq_all_props (__arr1, __arr2) {
  let arr = __arr1.concat(__arr2)
  console.log('arr.length', arr.length)
  let set = []
  let result = []
  arr.forEach(function (__obj) {
    /** Set each obj to a string. */
    let string = JSON.stringify(__obj)
    set.push(string)
  })
  set.filter(function (elem, index, self) {
    /** Use filter as a loop to push onto results array.
     * This is done to preserve prop types from original arrays */
    if (index === self.indexOf(elem)) {
      result.push(arr[index])
    }
  })
  return result
}


console.log(uniq_all_props(arr1, arr2))

Answer 5

Convert Array to JSON based on ID and then convert it into Array.

    const json = {};
    this.data.forEach(ele => {
      json[ele['id']] = ele;
    });
    console.log(Object.values(json));