Why does set( ) make this code run so much faster?

Question

I wrote some code for Project Euler Problem 35 :

#Project Euler: Problem 35

import time

start = time.time()

def sieve_erat(n):
    '''creates list of all primes < n'''
    x = range(2,n)
    b = 0
    while x[b] < int(n ** 0.5) + 1:
        x = filter(lambda y: y % x[b] != 0 or y == x[b], x)
        b += 1
    else:
        return x

def circularPrimes(n):
    '''returns # of circular primes below n'''
    count = 0
    primes = sieve_erat(n)
    b = set(primes)
    for prime in primes:
        inc = 0
        a = str(prime)
        while inc < len(a):
            if int(a) not in b:
                break
            a = a[-1] + a[0:len(a) - 1]
            inc += 1
        else:
            count += 1
    else:
        return count

print circularPrimes(1000000)
elapsed = (time.time() - start)
print "Found in %s seconds" % elapsed

I am wondering why this code (above) runs so much faster when I set b = set(primes) in the circularPrimes function. The running time for this code is about 8 seconds. Initially, I did not set b = set(primes) and my circularPrimes function was this:

def circularPrimes(n):
    '''returns # of circular primes below n'''
    count = 0
    primes = sieve_erat(n)
    for prime in primes:
        inc = 0
        a = str(prime)
        while inc < len(a):
            if int(a) not in primes:
                break
            a = a[-1] + a[0:len(a) - 1]
            inc += 1
        else:
            count += 1
    else:
        return count

My initial code (without b = set(primes) ) ran so long that I didn't wait for it to finish. I am curious as to why there is such a large discrepancy in terms of running time between the two pieces of code as I do not believe that primes would have had any duplicates that would have made iterating through it take so much longer that iterating through set(primes) . Maybe my idea of set( ) is wrong. Any help is welcome.

Answer 1

I believe the culprit here is if int(a) not in b: . Sets are implemented internally as hashtables, meaning that checking for membership is significantly less expensive than with a list (since you just need to check for collision).

You can check out the innards of sets here .