How can natural numbers be represented to offer constant time addition?

Question

Cirdec's answer to a largely unrelated question made me wonder how best to represent natural numbers with constant-time addition, subtraction by one, and testing for zero.

Why Peano arithmetic isn't good enough:

Suppose we use

data Nat = Z | S Nat

Then we can write

Z + n = n
S m + n = S(m+n)

We can calculate m+n in O(1) time by placing mr debits (for some constant r ), one on each S constructor added onto n . To get O(1) isZero , we need to be sure to have at most p debits per S constructor, for some constant p . This works great if we calculate a + (b + (c+...)) , but it falls apart if we calculate ((...+b)+c)+d . The trouble is that the debits stack up on the front end.

One option

The easy way out is to just use catenable lists, such as the ones Okasaki describes, directly. There are two problems:

1. O(n) space is not really ideal.

2. It's not entirely clear (at least to me) that the complexity of bootstrapped queues is necessary when we don't care about order the way we would for lists.

Answer 1

As far as I know, Idris (a dependently-typed purely functional language which is very close to Haskell) deals with this in a quite straightforward way. Compiler is aware of Nat s and Fin s (upper-bounded Nat s) and replaces them with machine integer types and operations whenever possible, so the resulting code is pretty effective. However, that's not true for custom types (even isomorphic ones) as well as for compilation stage (there were some code samples using Nat s for type checking which resulted in exponential growth in compile-time, I can provide them if needed).

In case of Haskell, I think a similar compiler extension may be implemented. Another possibility is to make TH macros which would transform the code. Of course, both of options aren't easy.

Answer 2

My understanding is that in basic computer programming terminology the underlying problem is you want to concatenate lists in constant time. The lists don't have cheats like forward references, so you can't jump to the end in O(1) time, for example.

You can use rings instead, which you can merge in O(1) time, regardless if a+(b+(c+...)) or ((...+c)+b)+a logic is used. The nodes in the rings don't need to be doubly linked, just a link to the next node.

Subtraction is the removal of any node, O(1), and testing for zero (or one) is trivial. Testing for n > 1 is O(n), however.

If you want to reduce space, then at each operation you can merge the nodes at the insertion or deletion points and weight the remaining ones higher. The more operations you do, the more compact the representation becomes! I think the worst case will still be O(n), however.

Answer 3

We know that there are two "extremal" solutions for efficient addition of natural numbers:

1. Memory efficient, the standard binary representation of natural numbers that uses O(log n) memory and requires O(log n) time for addition. (See also Chapter "Binary Representations" in the Okasaki's book .)

2. CPU efficient which use just O(1) time. (See Chapter "Structural Abstraction" in the book.) However, the solution uses O(n) memory as we'd represent natural number n as a list of n copies of () .

   I haven't done the actual calculations, but I believe for the O(1) numerical addition we won't need the full power of O(1) FIFO queues, it'd be enough to bootstrap standard list [] (LIFO) in the same way. If you're interested, I could try to elaborate on that.

The problem with the CPU efficient solution is that we need to add some redundancy to the memory representation so that we can spare enough CPU time. In some cases, adding such a redundancy can be accomplished without compromising the memory size (like for O(1) increment/decrement operation). And if we allow arbitrary tree shapes, like in the CPU efficient solution with bootstrapped lists, there are simply too many tree shapes to distinguish them in O(log n) memory.

So the question is: Can we find just the right amount of redundancy so that sub-linear amount of memory is enough and with which we could achieve O(1) addition? I believe the answer is no :

Let's have a representation+algorithm that has O(1) time addition. Let's then have a number of the magnitude of m -bits, which we compute as a sum of 2^k numbers, each of them of the magnitude of (mk) -bit. To represent each of those summands we need (regardless of the representation) minimum of (mk) bits of memory, so at the beginning, we start with (at least) (mk) 2^k bits of memory. Now at each of those 2^k additions, we are allowed to preform a constant amount of operations, so we are able to process (and ideally remove) total of C 2^k bits. Therefore at the end, the lower bound for the number of bits we need to represent the outcome is (mkC) 2^k bits. Since k can be chosen arbitrarily, our adversary can set k=mC-1 , which means the total sum will be represented with at least 2^(mC-1) = 2^m/2^(C+1) ∈ O(2^m) bits. So a natural number n will always need O(n) bits of memory!