Python output list of dictionary in specific format

Question

Hello I am new to python this may be something small but I am finding it difficult on how to get output in a specific format.

I have the following list of dict being returned

vpcs = [{'ResourceOwnerId': '111111111111', 'ResourceType': 'vpc', 'ResourceId': 'vpc-aaaaa'},
        {'ResourceOwnerId': '222222222222', 'ResourceType': 'vpc', 'ResourceId': 'vpc-ccccc'}
        {'ResourceOwnerId': '111111111111', 'ResourceType': 'vpc', 'ResourceId': 'vpc-ddddd'}]

Below is the snippet from my script

for vpc in vpcs:
        acc_data = {
            "account_number": vpc["ResourceOwnerId"],
            "vpc": [vpc['ResourceId']]
        }
        acc_list.append(acc_data)

print(acc_list)

The output of which is following

        [
            {'account_number': '111111111111', 'vpc': ['vpc-aaaaa']},
            {'account_number': '222222222222', 'vpc': ['vpc-ccccc']},
            {'account_number': '111111111111', 'vpc': ['vpc-ddddd']}
        ]

Whereas I want an output like this

        [
            {'account_number': '111111111111', 'vpc': ['vpc-aaaaa', 'vpc-ddddd']},
            {'account_number': '222222222222', 'vpc': ['vpc-ccccc']}
        ]

ie "account_number" which are same then those vpcs should be appended instead of creating a new entry. Any help would be greatly appreciated thank you

Answer 1

Just do it in two phases. Use a dictionary to build up the unique lists, then convert to a list of dictionaries:

from collections import defaultdict

vpcs = [{'ResourceOwnerId': '111111111111', 'ResourceType': 'vpc', 'ResourceId': 'vpc-aaaaa'},
        {'ResourceOwnerId': '222222222222', 'ResourceType': 'vpc', 'ResourceId': 'vpc-ccccc'},
        {'ResourceOwnerId': '111111111111', 'ResourceType': 'vpc', 'ResourceId': 'vpc-ddddd'}]

accum = defaultdict(list)
for vpc in vpcs:
    accum[vpc['ResourceOwnerId']].append( vpc['ResourceId'] )

print(accum)

acc_list = []
for k, v in accum.items():
    acc_list.append(
        {
            "account_number": k,
            "vpc": v
        }
    )

print(acc_list)

Output:

defaultdict(<class 'list'>, {'111111111111': ['vpc-aaaaa', 'vpc-ddddd'], '222222222222': ['vpc-ccccc']})
[{'account_number': '111111111111', 'vpc': ['vpc-aaaaa', 'vpc-ddddd']}, {'account_number': '222222222222', 'vpc': ['vpc-ccccc']}]

Or, for those who like one-liners:

...
acc_list = [ { "account_number": k, "vpc": v } for k, v in accum.items()]
print(acc_list)

Answer 2

You can use dict.setdefault to make create the basic sub-dicts with an empty list when the key is seen the first time. Then just append. The result will be in the dict.values() :

vpcs = [{'ResourceOwnerId': '111111111111', 'ResourceType': 'vpc', 'ResourceId': 'vpc-aaaaa'},
        {'ResourceOwnerId': '222222222222', 'ResourceType': 'vpc', 'ResourceId': 'vpc-ccccc'},
        {'ResourceOwnerId': '111111111111', 'ResourceType': 'vpc', 'ResourceId': 'vpc-ddddd'}]

groups = {}

for d in vpcs:
    k = d['ResourceOwnerId']
    groups.setdefault(k, {'account_number':k, 'vpc':[]} )['vpc'].append(d['ResourceId'])
    
list(groups.values())

This will give you:

[{'account_number': '111111111111', 'vpc': ['vpc-aaaaa', 'vpc-ddddd']},
 {'account_number': '222222222222', 'vpc': ['vpc-ccccc']}]

Answer 3

    from copy import deepcopy # create a copy of a list.
    # Your code snippet
    for vpc in vpcs:
        acc_data = {
            "account_number": vpc["ResourceOwnerId"],
            "vpc": [vpc['ResourceId']]
        }
        acc_list.append(acc_data)

    # Add this
    new_acc_list = deepcopy(acc_list)
    previous_acc_numbers = {} # int: index

    for acc_data in acc_list:
        acc_number, acc_vpc = acc_data.get(account_number), acc_data.get("vpc")
        if acc_number in previous_acc_numbers:
            new_acc_list[previous_acc_numbers.get(acc_number)].get("vpc").append(acc_data.get("vpc"))
    
    print(acc_list)

Answer 4

Try creating a dictionary mapping account number to the vpcs. Then transform this dictionary:

dct = dict()
for vpc in vpcs:
    if vpc["ResourceOwnerId"] not in dct:
        dct[vpc["ResourceOwnerId"]] = list()
    dct[vpc["ResourceOwnerId"]].append(vpc["ResourceId"])

output = [{"account_number": k, "vpc": v} for k,v in dct.items()]

>>> output
[{'account_number': '111111111111', 'vpc': ['vpc-aaaaa', 'vpc-ddddd']},
 {'account_number': '222222222222', 'vpc': ['vpc-ccccc']}]