Combining identical values against a list of dictionaries in Python

Question

I am building a table using python. The rows to the table are contained in a list of a dictionaries, like so:

table = [{ 'User' : 'johndoe', 'Profile' : 'Production', 'Risk' : 'No MFA', 'Action': 'Disable account and notify user' }, 
         { 'User' : 'janedoe', 'Profile' : 'Production', 'Risk' : 'Expired Password', 'Action': 'Reset password and notify user' },
         { 'User' : 'cookiedoe', 'Profile' : 'Non-Production', 'Risk' : 'Expired key', 'Action': 'Delete old key and notify user' },]

I run a function to build the row each time I detect a non-compliant condition. My goal is to combine rows where 'User' and 'Profile' are identical. For instance, if johndoe in Production also had an expired password, I want his dictionary to look thusly:

{ 'User' : 'johndoe', 'Profile' : 'Production', 'Risk' : 'No MFA, Expired password', Action: 'Disable account and notify user\n Reset password and notify user' }

Each current row lives in a local dictionary called 'row'. This is how I'm trying to accomplish this now:

for past_row in table:

     past_user = past_row['User']
     row_user = row['User']
     past_profile = past_row['Profile']
     row_profile = row['Profile']

     if past_user == row_user and past_profile == row_profile:

           past_row['Risk'] = "%s, %s" %(past_row['Risk'], row['Risk'])
           past_row['Action'] = "%s\n %s" %(past_row['Action'], row['Action'])

My Python script just runs endlessly, without ever finishing. I feel like I'm examining my past rows inefficiently, and I'm fairly young in my Pythonic education. Googling in circles is just making my logic more convoluted. Can someone set me straight?

Answer 1

Ok, I've come up with a quick solution. It can be done more efficiently, however, keeping in mind you're a beginner in Python, I placed my bets on readability. Here it goes (the script is executable as is):

# Import default-dictionary.
from collections import defaultdict

# Minimal sample, we want to merge johndoe.
table = [{ 'User' : 'johndoe', 'Profile' : 'Production', 'Risk' : 'No MFA', 'Action': 'Disable account and notify user' },
         { 'User' : 'johndoe', 'Profile' : 'Production', 'Risk' : 'Expired Password', 'Action': 'Reset password and notify user' },
         { 'User' : 'cookiedoe', 'Profile' : 'Non-Production', 'Risk' : 'Expired key', 'Action': 'Delete old key and notify user' },]


# Inline function definition for combinining risks/actions
# from multiple profiles.
combine_risks = lambda ps: ', '.join([p['Risk'] for p in ps])
combine_actions = lambda ps: ', '.join([p['Action'] for p in ps])

# Merge multiple profile dicts into one.
def combine_profiles(profiles):
    return dict(
        User=profiles[0]['User'],
        Profile=profiles[0]['Profile'],
        Risk=combine_risks(profiles),
        Action=combine_actions(profiles)
    )

# Profile - indices mapping.
prof2ind = defaultdict(set)

# Enumerate profiles and save information about which profiles
# have matching User and Profile.
for index, row in enumerate(table):
    key = (row['User'], row['Profile'])
    prof2ind[key].add(index)

new_table = []

# Finally, build merged table.
# 'indices' is a set holding indices of matching profile-sets.
for indices in prof2ind.values():
    profiles = [table[i] for i in indices]
    new_table.append(combine_profiles(profiles))

# Check the result.
print(new_table)

This script is general in the sense that is works on multiple matching profiles, not only pairs.