Finding digits in powers of 2 fast

Question

The task is to search every power of two below 2^10000, returning the index of the first power in which a string is contained. For example if the given string to search for is "7" the program will output 15, as 2^15 is the first power to contain 7 in it.

I have approached this with a brute force attempt which times out on ~70% of test cases.

for i in range(1,9999):
    if search in str(2**i):
        print i
        break

How would one approach this with a time limit of 5 seconds?

Answer 1

Try not to compute 2^i at each step.

pow = 1
for i in xrange(1,9999):
    if search in str(pow):
        print i
        break
    pow *= 2

You can compute it as you go along. This should save a lot of computation time.

Using xrange will prevent a list from being built, but that will probably not make much of a difference here.

in is probably implemented as a quadratic string search algorithm. It may (or may not, you'd have to test) be more efficient to use something like KMP for string searching.

Answer 2

A faster approach could be computing the numbers directly in decimal

def double(x):
    carry = 0
    for i, v in enumerate(x):
        d = v*2 + carry
        if d > 99999999:
            x[i] = d - 100000000
            carry = 1
        else:
            x[i] = d
            carry = 0
    if carry:
        x.append(carry)

Then the search function can become

def p2find(s):
    x = [1]
    for y in xrange(10000):
        if s in str(x[-1])+"".join(("00000000"+str(y))[-8:]
                                   for y in x[::-1][1:]):
            return y
        double(x)
    return None

Note also that the digits of all powers of two up to 2^10000 are just 15 millions, and searching the static data is much faster. If the program must not be restarted each time then

def p2find(s, digits = []):
    if len(digits) == 0:
        # This precomputation happens only ONCE
        p = 1
        for k in xrange(10000):
            digits.append(str(p))
            p *= 2
    for i, v in enumerate(digits):
        if s in v: return i
    return None

With this approach the first check will take some time, next ones will be very very fast.

Answer 3

Compute every power of two and build a suffix tree using each string. This is linear time in the size of all the strings. Now, the lookups are basically linear time in the length of each lookup string.

I don't think you can beat this for computational complexity.

Answer 4

There are only 10000 numbers. You don't need any complex algorithms. Simply calculated them in advance and do search. This should take merely 1 or 2 seconds.

powers_of_2 = [str(1<<i) for i in range(10000)]

def search(s):
    for i in range(len(powers_of_2)):
        if s in powers_of_2[i]:
            return i

Answer 5

Try this

twos = []
twoslen = []
two = 1
for i in xrange(10000):
    twos.append(two)
    twoslen.append(len(str(two)))
    two *= 2

tens = []
ten = 1
for i in xrange(len(str(two))):
    tens.append(ten)
    ten *= 10

s = raw_input()
l = len(s)
n = int(s)

for i in xrange(len(twos)):
    for j in xrange(twoslen[i]):
        k = twos[i] / tens[j]
        if k < n: continue
        if (k - n) % tens[l] == 0:
            print i
            exit()

The idea is to precompute every power of 2, 10 and and also to precompute the number of digits for every power of 2. In this way the problem is reduces to finding the minimum i for which there exist aj such that after removing the last j digits from 2 ** i you obtain a number which ends with n or expressed as a formula (2 ** i / 10 ** j - n) % 10 ** len(str(n)) == 0.

Answer 6

A big problem here is that converting a binary integer to decimal notation takes time quadratic in the number of bits (at least in the straightforward way Python does it). It's actually faster to fake your own decimal arithmetic, as @6502 did in his answer.

But it's very much faster to let Python's decimal module do it - at least under Python 3.3.2 (I don't know how much C acceleration is built in to Python decimal versions before that). Here's code:

class S:
    def __init__(self):
        import decimal
        decimal.getcontext().prec = 4000  # way more than enough for 2**10000
        p2 = decimal.Decimal(1)
        full = []
        for i in range(10000):
            s = "%s<%s>" % (p2, i)
            ##assert s == "%s<%s>" % (str(2**i), i)
            full.append(s)
            p2 *= 2
        self.full = "".join(full)

    def find(self, s):
        import re
        pat = s + "[^<>]*<(\d+)>"
        m = re.search(pat, self.full)
        if m:
            return int(m.group(1))
        else:
            print(s, "not found!")

and sample usage:

>>> s = S()
>>> s.find("1")
0
>>> s.find("2")
1
>>> s.find("3")
5
>>> s.find("65")
16
>>> s.find("7")
15
>>> s.find("00000")
1491
>>> s.find("666")
157
>>> s.find("666666")
2269
>>> s.find("66666666")
66666666 not found!

s.full is a string with a bit over 15 million characters. It looks like this:

>>> print(s.full[:20], "...", s.full[-20:])
1<0>2<1>4<2>8<3>16<4 ... 52396298354688<9999>

So the string contains each power of 2, with the exponent following a power enclosed in angle brackets. The find() method constructs a regular expression to search for the desired substring, then look ahead to find the power.

Playing around with this, I'm convinced that just about any way of searching is "fast enough". It's getting the decimal representations of the large powers that sucks up the vast bulk of the time. And the decimal module solves that one.