Python Remove duplicates and original from nested list based on specific key

Question

I m trying to delete all duplicates & original from a nested list based on specific column.

Example

list = [['abc',3232,'demo text'],['def',9834,'another text'],['abc',0988,'another another text'],['poi',1234,'text']]

The key column is the first (abc, def, abc) and based on this I want to remove any item (plus the original) which has the same value with the original.

So the new list should contain:

newlist = [['def',9834,'another text'],['poi',1234,'text']]

I found many similar topics but not for nested lists... Any help please?

Answer 1

You can construct a list of keys

keys = [x[0] for x in list]

and select only those records for which the key occurs exactly once

newlist = [x for x in list if keys.count(x[0]) == 1]

Answer 2

Using a list comprehension.

Demo:

l = [['abc',3232,'demo text'],['def',9834,'another text'],['abc', 988,'another another text'],['poi',1234,'text']]
checkVal = [i[0] for i in l]
print( [i for i in l if not checkVal.count(i[0]) > 1 ] )

Output:

[['def', 9834, 'another text'], ['poi', 1234, 'text']]

Answer 3

Using collections.defaultdict for an O(n) solution:

L = [['abc',3232,'demo text'],
     ['def',9834,'another text'],
     ['abc',988,'another another text'],
     ['poi',1234,'text']]

from collections import defaultdict

d = defaultdict(list)

for key, num, txt in L:
    d[key].append([num, txt])

res = [[k, *v[0]] for k, v in d.items() if len(v) == 1]

print(res)

[['def', 9834, 'another text'],
 ['poi', 1234, 'text']]

Answer 4

Use collections.Counter :

from collections import Counter

lst = [['abc',3232,'demo text'],['def',9834,'another text'],['abc',988,'another another text'],['poi',1234,'text']]

d = dict(Counter(x[0] for x in lst))
print([x for x in lst if d[x[0]] == 1])

# [['def', 9834, 'another text'], 
#  ['poi', 1234, 'text']]

Also note that you shouldn't name your list as list as it shadows the built-in list .