Python list comprehension Remove duplicates

Question

I want unique elements in hubcode_list . I can do it by

hubcode_alarm_obj = HubAlarm.objects.all()
for obj in hubcode_alarm_obj:
    hubcode = obj.hubcode
    if hubcode not in hubcode_list:
        hubcode_list.append(hubcode)

I want to use list comprehension. I'm trying this but it says hubcode_list is undefined, which is the correct bug. How can I only add unique elements in hubcode_list

hubcode_alarm_obj = HubAlarm.objects.all()
hubcode_list = [obj.hubcode for obj in hubcode_alarm_obj if obj.hubcode not in hubcode_list]

I can also do it by:-

hubcode_list = list(set([obj.hubcode for obj in hubcode_alarm_obj]))

But set is again another operation. Can I do it by using if-statement in list-comprehension?

Answer 1

Since you are using django, let the database do the filtering for you. It will be (by far) the fastest option:

objects = HubAlarm.objects.exclude(hubcode__in=hubcode_list)

See the documentation for more.

Answer 2

Your answer is no. List comprehension creates a completely new list all at one shot, so you can't check to see if something exists:

>>> lst = [1, 2, 2, 4, 5, 5, 5, 6, 7, 7, 3, 3, 4]
>>> new = [item for item in lst if item not in new]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'new' is not defined
>>> 

Therefore, the preferred one-liner is to use list(set(...)) .

>>> lst = [1, 2, 2, 4, 5, 5, 5, 6, 7, 7, 3, 3, 4]
>>> list(set(lst))
[1, 2, 3, 4, 5, 6, 7]
>>> 

However, in a list of lists, this would not work:

>>> lst = [[1, 2], [1, 2], [3, 5], [6, 7]]
>>> list(set(lst))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'list'
>>> 

In that case, use the standard for loop and .append() .