从 HashMap 条目列表中删除重复项

Question

I have a List<HashMap<String,Object>> which represents a database where each list record is a database row.

I have 10 columns in my database. There are several rows where the values of 2 particular columns are equals. I need to remove the duplicates from the list after the list is updated with all the rows from database.

What is the efficient way?

FYI - I am not able to do distinct while querying the database, because the GroupName is added at a later stage to the Map after the database is loaded. And since Id column is not primary key, once you add GroupName to the Map . You will have duplicates based on Id + GroupName combination!

Hope my question makes sense. Let me know if we need more clarification.

Answer 1

1. create a Comparator that compares HashMaps, and compares them by comparing the key/value pairs you are interested in.
2. use Collections.sort(yourlist, yourcomparator) ;
3. Now all maps that are similar to each other, based on your comparator, are adjacent in the list.
4. Create a new list.
5. Iterate through your first list, keeping track of what you saw last. If the current value is different than the last, add this to your new list.
6. You new list should contain no duplicates according to your comparator.

The cost of iterating through the list is O(n). Sorting is O(n log n). So this algorithm is O(n log n).

We could also sort on-the-fly by using a TreeSet with that comparator. Inserts are O(log n). And we have to do this n times. So we get O(n log n).

Answer 2

I was having same condition, after lot of search, finally below solution worked for me.

1. Create POJO, override equal () & hashCode ().
2. equality should be on all data members with .equal comparison.
3. Hashcode calculated as Objects.hash(filed1, field2, ...) ;
4. Iterate over list and construct POJO object and add it to HashSet

Below is code -

/* (non-Javadoc)
 * @see java.lang.Object#equals(java.lang.Object)
 */
@Override
public boolean equals(Object obj) {
    if( this == obj) {
        return true;
    }
    if( null == obj || getClass() != obj.getClass()) {
        return false;
    }

    Pojo rest = (Pojo) obj;
    return field1.equals(rest.field1) && field2.equals(rest.field2) && field3.equals(rest.field3); 
}

/* (non-Javadoc)
 * @see java.lang.Object#hashCode()
 */
@Override
public int hashCode() {
    return Objects.hash(field1, field2, field3);
}

Set<Pojo> pathSet = new HashSet<>();
//Iterate over list and construct pojo object here
Pojo pojo = new Pojo(value1, value2, value3)
pathSet.add(pojo)

Thanks,

Answer 3

I have taken an Employee class and created Map with Integer,Employee object as key-value pair here is my Map

Map<Integer,Employee> map = new HashMap<Integer,Employee>();

Employee class is a bean class and it has properties like name,id,designation; map allow unique keys. but if you don't want to allow duplicate values in your map you have to over ride equals method in bean class.

@Override
public boolean equals(Object object){
    if (object == null) return false;
    if (object == this) return true;
    if (this.getClass() != object.getClass())return false;
    Employee employee = (Employee)object;
    if(this.hashCode()== employee.hashCode())return true;
   return false;
}  

and while adding key-value to Map you have to use contains method

if(!map.containsValue(map.get(id))){
   map.put(id,employee);
}

containsValue internally calls equals() method and hence you over ride equals method it will check every value(object) with previous objects and if hash codes are same it returns true means both are same objects.