How can I use list comprehension to separate values in a dictionary?

Question

name=[]
age=[]
address=[]
...
for line in pg:
    for key,value in line.items():
        if key == 'name':
          name.append(value)
        elif key == 'age':
          age.append(value)
        elif key == 'address':
          address.append(value)
        .
        .
        .

Is it possible to use list comprehension for above code because I need to separate lots of value in the dict? I will use the lists to write to a text file.

Source Data:

a = [{'name': 'paul', 'age': '26.', 'address': 'AU', 'gender': 'male'},
     {'name': 'mei', 'age': '26.', 'address': 'NY', 'gender': 'female'},
     {'name': 'smith', 'age': '16.', 'address': 'NY', 'gender': 'male'},
     {'name': 'raj', 'age': '13.', 'address': 'IND', 'gender': 'male'}]

Answer 1

I don't think list comprehension will be a wise choice because you have multiple lists. Instead of making multiple lists and appending to them the value if the key matches you can use defaultdict to simplify your code.

from collections import defaultdict

result = defaultdict(list)
for line in pg:
    for key, value in line.items():
        result[key].append(value)

You can get the name list by using result.get('name')

['paul', 'mei', 'smith', 'raj']

Answer 2

This probably won't work the way you want: Your'e trying to assign the three different lists, so you would need three different comprehensions. If your dict is large, this would roughly triple your execution time.

Something straightforward, such as

name = [value for for key,value in line.items() if key == "name"]

seems to be what you'd want... three times.

Answer 3

You can proceed as:

pg=[{"name":"name1","age":"age1","address":"address1"},{"name":"name2","age":"age2","address":"address2"}]

name=[v for line in pg for k,v in line.items() if k=="name"]
age=[v for line in pg for k,v in line.items() if k=="age"]
address=[v for line in pg for k,v in line.items() if k=="address"]

Answer 4

Here is a really simple solution if you want to use pandas :

import pandas as pd

df = pd.DataFrame(a)

name = df['name'].tolist()
age = df['age'].tolist()
address = df['address'].tolist()

print(name)
print(age)
print(address)

Output:

['paul', 'mei', 'smith', 'raj']
['26.', '26.', '16.', '13.']
['AU', 'NY', 'NY', 'IND']

Additionally, if your end result is a text file, you can skip the list creation and write the DataFrame (or parts thereof) directly to a CSV with something as simple as:

df.to_csv('/path/to/output.csv')

Answer 5

In continuation with Vishal's answer, please dont use defaultdict. Using defaultdict is a very bad practice when you want to catch keyerrors. Please use setdefault.

results = dict()
for line in pg:
    for key, value in line.items():
        result.setdefault(key, []).append(value)

Output

{
     'name': ['paul', 'mei', 'smith', 'raj'],
     'age': [26, 26, 26, 13],
      ...
}

However, note that if all dicts in pg dont have the same keys, you will lose the relation/correspondence between the items in the dict