Remove duplicates from a list of nested dictionaries

Question

I'm writing my first python program to manage users in Atlassian On Demand using their RESTful API. I call the users/search?username= API to retrieve lists of users, which returns JSON. The results is a list of complex dictionary types that look something like this:

[
        {
            "self": "http://www.example.com/jira/rest/api/2/user?username=fred",
            "name": "fred",
            "avatarUrls": {
                "24x24": "http://www.example.com/jira/secure/useravatar?size=small&ownerId=fred",
                "16x16": "http://www.example.com/jira/secure/useravatar?size=xsmall&ownerId=fred",
                "32x32": "http://www.example.com/jira/secure/useravatar?size=medium&ownerId=fred",
                "48x48": "http://www.example.com/jira/secure/useravatar?size=large&ownerId=fred"
            },
            "displayName": "Fred F. User",
            "active": false
        },
        {
            "self": "http://www.example.com/jira/rest/api/2/user?username=andrew",
            "name": "andrew",
            "avatarUrls": {
                "24x24": "http://www.example.com/jira/secure/useravatar?size=small&ownerId=andrew",
                "16x16": "http://www.example.com/jira/secure/useravatar?size=xsmall&ownerId=andrew",
                "32x32": "http://www.example.com/jira/secure/useravatar?size=medium&ownerId=andrew",
                "48x48": "http://www.example.com/jira/secure/useravatar?size=large&ownerId=andrew"
            },
            "displayName": "Andrew Anderson",
            "active": false
        }
    ]

I'm calling this multiple times and thus getting duplicate people in my results. I have been searching and reading but cannot figure out how to deduplicate this list. I figured out how to sort this list using a lambda function. I realize I could sort the list, then iterate and delete duplicates. I'm thinking there must be a more elegant solution.

Thank you!

Answer 1

The usernames are unique, right?

Does it have to be a list ? Seems like an easy solution would be to make it a dict of dict s instead. Use the usernames as keys, and only the most recent version will be present.

If the values have to be ordered, there is an OrderedDict type you could look into: http://docs.python.org/2/library/collections.html#collections.OrderedDict

Answer 2

Let say it is what you got,

JSON = [
        {

            "name": "fred",
...
},
        {

            "name": "peter",
...
},
        {

            "name": "fred",
...
},

Convert this list of dict to a dict of dict will remove the duplicate, like so:

r = dict([(user['name'], user) for user in JSON])

In r you will only find one record of fred and peter each.