将循环写入pythonic方式

Question

I have a dict array that stores audits for tickets. Each audit has an information of user_id , date that happens changes and a list of events and each event has a few attributes like type , field name , among others.

Based on those informations, I need to extract events information based on date and convert to another dict. Note: I need to keep only the last event for each field_name .

I've wrote a "super" loop that does what I need but this code looks pretty weird and not optmized:

dict sample:

data = {
    "audits": [
        "id": 1234,
            "ticket_id": 1111,
            "created_at": "2019-04-07T01:09:40Z",
            "author_id": 9876543,           
            "events": [{
                    "id": 1234,
                    "type": "Random"
                },
                {
                    "id": 765456,
                    "type": "Create",
                    "value": "Lovely form",
                    "field_name": "subject"
                },              
                {
                    "id": 356765,
                    "type": "Create",
                    "value": None,
                    "field_name": "priority"
                },              
                {
                    "id": 2345432,
                    "type": "Change",                   
                    "value": "normal",
                    "field_name": "priority",
                    "previous_value": None
                }
            ]
        }
    ]
}

code:

field_history = []

for audit in data['audits']:
    user_id = audit['author_id']
    updated = audit['created_at']

    base_info = {
        'user_id': user_id,
        'updated': updated
    }

    # Iterate to get distinct value (last found on dict)
    fields = [d for d in audit['events'] if (d['type'] == 'Create' or d['type'] == 'Change') and d['field_name'] != 'tags']        
    updated_fields = [] # this list is being used to keep history by updated
    for field in fields:
        distincts = [d for d in audit['events'] if d.get('field_name', '') == field['field_name']]        
        distinct = distincts[-1]
        # remove older values and keep only the last one found on list
        updated_fields[:] = [d for d in updated_fields if d['updated'] == updated and d.get('field_name') != distinct['field_name']]
        updated_fields.append({**base_info, **distinct}) # add always the last element on list

    field_history = field_history + updated_fields

What is the proper way to write this loop making it optimized to handle large datasets?

Answer 1

I like to start by making some simple functions to handle the transformations and filtering to allow the top level to remain clean:

def event_valid(event):
    return (
        event['type'] in ('Create', 'Change')
        and event['field_name'] not in ('tags',)
    )

events = [event for event in audit['events'] if event_valid(event)]

# Assuming the list is ordered... If not then sort it before next statement
# This trick filters to only the latest event for each distinct field_name
events = {
    event['field_name']: event for event in events
}.values()

return {
    'user_id': audit['author_id'],
    'updated': audit['created_at'],
    'events': events,
}