Working with big numbers in Python and writing them to file

Question

I'm trying to find an efficient way to do the following in Python:

   a = 12345678901234567890123456**12345678
   f = open('file', 'w')
   f.write(str(a))
   f.close()

The calculation of the power takes about 40 minutes while one thread is utilized. Is there a quick and easy way to spread this operation over multiple threads?

As the number is quite huge, I think the string function isn't quite up to the task - it's been going for almost three hours now. I need the number to end up in a text file. Any ideas on how to better accomplish this?

Answer 1

I would like to give a lavish ;-) answer, but don't have the time now. Elaborating on my comment, the decimal module is what you really want here. It's much faster at computing the power, and very very much faster to convert the result to a decimal string:

>>> import decimal

You need to change its internals so that it avoids floating point, giving it more than enough internal digits to store the final result. We want exact integer arithmetic here, not rounded floating-point. So we fiddle things so decimal uses as much precision as it's capable of using, and tell it to raise the "Inexact" exception if it ever loses information to rounding. Note that you need a 64-bit version of Python for decimal to be capable of using enough precision to hold the exact result in your example:

>>> import decimal
>>> c = decimal.getcontext()
>>> c.prec = decimal.MAX_PREC
>>> c.Emax = decimal.MAX_EMAX
>>> c.Emin = decimal.MIN_EMIN
>>> c.traps[decimal.Inexact] = 1

Now create a Decimal for the base:

>>> base = decimal.Decimal(12345678901234567890123456)
>>> base
Decimal('12345678901234567890123456')

And raise to the power - the exponent will automatically be converted to Decimal , because the base is already Decimal :

>>> x = base ** 12345678

That takes less than a minute on my box! The reasons for that are involved. It's not really because it's working in base 10, but because the person who wrote the decimal module implemented "advanced" algorithms for doing very large multiplications.

Now convert to a string. Because it's already stored in a variant of base 10, converting to a decimal string goes very fast (a few seconds on my box, just because the string has hundreds of millions of digits):

>>> y = str(x)
>>> len(y)
309771765

And, for sanity, let's just look at the last 10, and first 10, digits:

>>> y[-10:]
'6044706816'
>>> y[:10]
'2759594879'

As @StefanPochmann noted in a comment, the last 10 digits can be obtained very quickly with native ints by using modular (3-argument) pow() :

>>> pow(int(base), 12345678, 10**10)
6044706816

Which matches the last 10 digits of the string above. For the first 10 digits, we can use decimal again but with much less precision, which will cause it (you'll just to have trust me on this) to use a different approach under the covers:

>>> c.prec = 12
>>> c.traps[decimal.Inexact] = 0  # don't trap on rounding!
>>> base ** 12345678
Decimal('2.75959487945E+309771764')

Rounding that back to 10 digits matches the earlier result, and the exponent is consistent with the length of y too.