从熊猫列表的字典中删除重复项

Question

I'm trying to extract dictionary from dataframe with no duplicates.

Here is the dataframe:

{'Country': {0: 'Japan', 1: 'China', 2: 'USA', 3: 'Russia', 4: 'Japan', 
5: 'Japan', 6: 'China'}, 'Port': {0: 'Yokohama', 1: 'Ningbo', 2: 
'Baltimore', 3: 'Moscow', 4: 'Tokyo', 5: 'Tokyo', 6: 'Shanghai'}}

I set the countries as keys and removed duplicates.Now I need to remove the duplicates from list

import pandas as pd
a ={'Country': {0: 'Japan', 1: 'China', 2: 'USA', 3: 'Russia', 4: 'Japan', 
5: 'Japan', 6: 'China'}, 'Port': {0: 'Yokohama', 1: 'Ningbo', 2: 
'Baltimore', 3: 'Moscow', 4: 'Tokyo', 5: 'Tokyo', 6: 'Shanghai'}}

a_dict=a.groupby(['Country'])['Port'].apply(list).to_dict()
print(a_dict)

Output:

{'China': ['Ningbo', 'Shanghai'], 'Japan': ['Yokohama', 'Tokyo', 
'Tokyo'], 'Russia': ['Moscow'], 'USA': ['Baltimore']}

Expected output:

{'China': ['Ningbo', 'Shanghai'], 'Japan': ['Yokohama', 'Tokyo'], 
'Russia': ['Moscow'], 'USA': ['Baltimore']}

Answer 1

Use drop_duplicates along with your code:

d = df.drop_duplicates().groupby(['Country'])['Port'].apply(list).to_dict()

print(d)
{'China': ['Ningbo', 'Shanghai'], 'Japan': ['Yokohama', 'Tokyo'], 
 'Russia': ['Moscow'], 'USA': ['Baltimore']}

Answer 2

GroupBy.apply with set

df.groupby('Country')['Port'].apply(set).map(list).to_dict()

If you don't care that your output is a dict of lists or dict of sets, this will simplify to

df.groupby('Country')['Port'].apply(set).to_dict()

---

defaultdict

from collections import defaultdict

d = defaultdict(set)
for c, p in zip(df['Country'], df['Port']):
    d[c].add(p)

{k: list(v) for k, v in d.items()}