Python what is the pythonic way of comparing 2 increasing in size lists?

Question

I have one list (list a) that increases based on size and has items removed from this list and put into another list (list b) that maintains what has already been checked.

The list that is maintaining (list b) what has been done needs to be compared with the list that has content (list a) that hasn't been checked already; what is the most efficient way to do this check in python? Even when the contents of each list can go well over 10k items.

while listA:
    for a in listA:
        #do something

        listB.append(url)
        listA.remove(url)

This is the furthest I can get.

Answer 1

Your code will produce inconsistent output. You shouldn't iterate and modify a list in the same loop context.

You could use listA as a Queue :

 while len(listA):
     element = listA.pop()
     #do something with element
     listB.add(element)

this code is consistent because even tough the list is being modified within the while loop there is no iterator created for listA . Only the len of the list is checked to make sure that the program terminates.

Answer 2

It depends on what you really need. To compare unique values from two lists you can try to use set data type :

a = xrange(11000)
b = xrange(0,11000,10)
diff = set(a).difference(set(b))
inter = set(a).intersection(set(b))

Also you can try to take a look at the difference_update that:

Update the set, removing elements found in others.