Python convert nested loop to list comprehension

Question

I have multi level list of dictionary from which I need to make two required list. Currently I can do it this way but wanted to make fully list comprehension way,

poi = [] 
nh = [] 
tr_poi_nh = response.xpath('string(//body)').re(r'window\.__WEB_CONTEXT__=(.*?);\(this')[0]

tr_poi_nh = json.loads(tr_poi_nh.replace("pageManifest", '"pageManifest"'))

for entry in tr_poi_nh['pageManifest']['hydrations']:   
    if entry['package'] == 'e374ol':       
        for value in entry['props']['initialAvailableFilters']:         
            if value['paramName'] =='distFrom':     
                poi = [x['normalized_name']  for x in value['poiSet']]   
            if value['paramName'] == 'zfn':                         
                nh = [y['label']  for y in value['choices']]

print(poi,nh)
    

Answer 1

I agree with gddc's comment. Code readability is highly wanted.

On another note, it looks like you're overriding poi and nh at with the suggested approach.

I'm guessing this will give you a more wanted approach:

poi = [] 
nh = [] 
tr_poi_nh = response.xpath('string(//body)').re(r'window\.__WEB_CONTEXT__=(.*?);\(this')[0]

tr_poi_nh = json.loads(tr_poi_nh.replace("pageManifest", '"pageManifest"'))

for entry in tr_poi_nh['pageManifest']['hydrations']:   
    if entry['package'] == 'e374ol':       
        for value in entry['props']['initialAvailableFilters']:         
            if value['paramName'] =='distFrom':     
                poi.extend([x['normalized_name']  for x in value['poiSet']])
            if value['paramName'] == 'zfn':                         
                nh.extend([y['label']  for y in value['choices']])

print(poi,nh)

By using extend you will add all of the values to your original list rather then override the entire list.

Answer 2

As gddc mentioned it is not always the case that a list comprehension will be advantageous. In this case it will have better performance because you will avoid calling append on each iteration, as explained by this answer . If you make good use of indentation you can still have a easily readable list comprehension.

The only problem is, you cannot return two different lists from a list comprehension, the closest you could get is to return a single list with (poi, nh) pairs, where poi and nh are lists.

Another consideration which is not clear in your question is if you want to append or extend each list.

• Appending will result in a list of lists, useful in some use cases, in which case you can definitely improve performance by building a list comprehension .
• Extending will result in a flat list, if you you only care about all elements regardless of the source. In this case, if the data is on the moderate-to-small size, you could be better off leaving the loop as it is and avoiding further steps to separate tuples (see below)

Anyway here is the list comprehension. You can tell it is basically the inner-most code first then all the for s and if s in order.

main_list = [
    (
        [x['normalized_name']  for x in value['poiSet']]
            if value['paramName'] == 'distFrom' else []
        ,
        [y['label']  for y in value['choices']]
            if value['paramName'] == 'znf' else []
    )
    for entry in tr_poi_nh['pageManifest']['hydrations']  
    if entry['package'] == 'e374ol'
    for value in entry['props']['initialAvailableFilters']
]

I would think about which is easiest from here, either to use main_list as it is and modify further code, or separate the tuples into two lists.

If you MUST have poi and nh as two separate lists, you now need two list comprehensions.

# to have lists of lists
poi = [x[0] for x in main_list]
nh = [x[1] for x in main_list]

# OR
# to have flat lists
poi = [e for x in main_list for e in x[0]]
nh = [e for x in main_list for e in x[1]]