如何在dict python 3.4的值列表中找到第一个唯一项

Question

Hello all I have a dict

dat = {
       '2018-01':['jack', 'jhon','mary','mary','jack'],
       '2018-02':['Oliver', 'Connor','mary','Liam','jack','Oliver'],
       '2018-03':['Jacob', 'jhon','Reece','mary','jack'],
       '2018-04':['George', 'jhon','mary','Alexander','Richard'],
}

I want the output like this:

    Output = {
              '2018-01':['jack','jhon','mary'],
              '2018-02':['Oliver', 'Connor','Liam'],
              '2018-03':['Jacob','Reece'],
              '2018-04':['George','Alexander','Richard'] 
}

I have my code which is a nested for loop inserting it to a list

lis = []
for key,value in dat.iteritems():   
    for va in value:
        if va not in lis:
            val = key,va
            lis.append(val)

But my dict "dat" has so many items in the values in that list. How can I do this with out nested for loop its consuming a lot of time.

Thanks in advance

Answer 1

What you are trying to do is this:

dat = {
       '2018-01':['jack', 'jhon','mary','mary','jack'],
       '2018-02':['Oliver', 'Connor','mary','Liam','jack','Oliver'],
       '2018-03':['Jacob', 'jhon','Reece','mary','jack'],
       '2018-04':['George', 'jhon','mary','Alexander','Richard'],
}

unique = set()
res = {}
for key, values in dat.items():
    res[key] = []
    for value in values:
        if value not in unique:
            res[key].append(value)
            unique.add(value)

which produces:

{'2018-01': ['jack', 'jhon', 'mary'], 
 '2018-02': ['Oliver', 'Connor', 'Liam'], 
 '2018-03': ['Jacob', 'Reece'], 
 '2018-04': ['George', 'Alexander', 'Richard']}

---

BUT

the order in dictionaries prior to Python version 3.7 could not be guaranteed and this makes the above code dangerous . The reason that is, is that with the same input you might end up having multiple different outputs.

To understand why take a look at this:

list1 = ['foo', 'bar', 'foobar']
list2 = ['bar']

1. If I use list1 to eliminate all duplicates I would end up with:

    list1 = ['foo', 'bar', 'foobar'] list2 = [] 

2. If I use list2 to eliminate all duplicates I would end up with:

    list1 = ['foo', 'foobar'] list2 = ['bar'] 

So depending on what I start with I end up having different results. With the dict from your example, what list you start with is any man's guess .

---

There is still hope however

because you can start with an OrderedDict (from collections ):

dat = OrderedDict([('2018-01', ['jack', 'jhon', 'mary', 'mary', 'jack']), 
                   ('2018-02', ['Oliver', 'Connor', 'mary', 'Liam', 'jack', 'Oliver']), 
                   ('2018-03', ['Jacob', 'jhon', 'Reece', 'mary', 'jack']), 
                   ('2018-04', ['George', 'jhon', 'mary', 'Alexander', 'Richard'])])

and then continue with the rest of the code as before.

Answer 2

Another take on @Ev. Kounis's approach using sets and OrderedDict (and pprint for sake of pretty printing):

import pprint
from collections import OrderedDict

dat = OrderedDict({
    '2018-01': ['jack', 'jhon', 'mary', 'mary', 'jack'],
    '2018-02': ['Oliver', 'Connor', 'mary', 'Liam', 'jack', 'Oliver'],
    '2018-03': ['Jacob', 'jhon', 'Reece', 'mary', 'jack'],
    '2018-04': ['George', 'jhon', 'mary', 'Alexander', 'Richard'],
})

exist = set()
output = OrderedDict()

for k, v in dat.items():
    output[k] = set(v) - exist
    exist.update(v)

pprint.pprint(output)

# OrderedDict([('2018-01', {'mary', 'jack', 'jhon'}),
#             ('2018-02', {'Connor', 'Oliver', 'Liam'}),
#             ('2018-03', {'Jacob', 'Reece'}),
#             ('2018-04', {'George', 'Alexander', 'Richard'})])

Answer 3

You can do something like this:

l=[]
for k,v in dat.items():
    dat[k] = list(set([i for i in v if i not in l]))
    l = l + v

now dat will be:

{
    '2018-01': ['jhon', 'mary', 'jack'],
    '2018-02': ['Oliver', 'Liam', 'Connor'],
    '2018-03': ['Jacob', 'Reece'],
    '2018-04': ['George', 'Alexander', 'Richard']
}

Answer 4

If you don't mind about the order in the list of values this can be a solution. Note the outpout of this solution may be different according to the version of Python. Indeed dict are guaranteed to be insertion ordered only from Python3.6.

dat = {
'2018-01': ['jack', 'jhon', 'mary', 'mary', 'jack'],
'2018-02': ['Oliver', 'Connor', 'mary', 'Liam', 'jack', 'Oliver'],
'2018-03': ['Jacob', 'jhon', 'Reece', 'mary', 'jack'],
'2018-04': ['George', 'jhon', 'mary', 'Alexander', 'Richard'],
}

s = set()
d = {}
for k,v in dat.items():
    d[k] = list(set(v) - s)
    s.update(d[k])

#{'2018-01': ['jack', 'jhon', 'mary'], '2018-02': ['Connor', 'Oliver', 'Liam'], '2018-03': ['Reece', 'Jacob'], '2018-04': ['Richard', 'Alexander', 'George']}

Answer 5

I think that what you need, I just edit your code

dat = {
       '2018-01':['jack', 'jhon','mary','mary','jack'],
       '2018-02':['Oliver', 'Connor','mary','Liam','jack','Oliver'],
       '2018-03':['Jacob', 'jhon','Reece','mary','jack'],
       '2018-04':['George', 'jhon','mary','Alexander','Richard'],
}

lis= dat.values()
lis = list(set([item for sublist in lis for item in sublist]))
out_val = []
for key,value in dat.iteritems():   
    res = []
    for i in value :
        if i in lis :
            res.append(i)
            lis.remove(i)
    out_val.append(res)

your_output=dict(zip( dat.keys(), out_val))

Output :

{'2018-01': ['jack', 'jhon', 'mary'], 
'2018-03': ['Jacob', 'Reece'], 
'2018-02': ['Oliver', 'Connor', 'Liam'], 
'2018-04': ['George', 'Alexander', 'Richard']}

Answer 6

Assuming the order is by the keys ['2018-01', '2018-02', '2018-03', '2018-04'] you could loop over the keys in that order, like this:

d = {'2018-01': ['jack', 'jhon', 'mary', 'mary', 'jack'],
     '2018-02': ['Oliver', 'Connor', 'mary', 'Liam', 'jack', 'Oliver'],
     '2018-03': ['Jacob', 'jhon', 'Reece', 'mary', 'jack'],
     '2018-04': ['George', 'jhon', 'mary', 'Alexander', 'Richard']}

result = {}
found = set()
for i in sorted(d):
    result[i] = list(set(d[i]).difference(found))
    found.update(d[i])

for i in sorted(result):
     print(i, result[i])

Output

2018-01 ['mary', 'jhon', 'jack']
2018-02 ['Oliver', 'Liam', 'Connor']
2018-03 ['Reece', 'Jacob']
2018-04 ['Alexander', 'Richard', 'George']

Answer 7

Try this.

tmp_list1 = []

for key,value in dat.iteritems():

    tmp_list2 = []

    dat[key] = list(set(value))

    for val in dat[key]:

        if val not in tmp_list1:

            tmp_list2.append(val)

    dat[key] = tmp_list2

    tmp_list1 = tmp_list1 + tmp_list2

print dat

Answer 8

import itertools
for i in d:
     d[i].sort()
     d[i] = list(i for i, _ in itertools.groupby(d[i]))

# Print the dict containing unique lists for keys.
for i in d:
     print(i, "->", d[i])