lazy mapping between integers and arrays permutations without generating all the permutations

Question

I want to generate a mapping between a set of integer arrays and integer values in python, eg for n=2 (size of the array) and m=2 (array item range), I want to have:

{
   0: [0, 0, 0],
   1: [0, 0, 1],
   2: [0, 1, 1],
   3: [1, 0, 0],
   ...
   7: [1, 1, 1]

}

One way to do this is to use the itertools.product and do the following:

from itertools import product
import numpy as np
n = 2
m = 3
a = list(product(np.arange(n), repeat=3))
a = list(map(lambda l: list(l), a))
b = dict(zip(np.arange(len(a)), a))

and b will be:

{0: [0, 0, 0], 1: [0, 0, 1], 2: [0, 1, 0], 3: [0, 1, 1], 4: [1, 0, 0], 5: [1, 0, 1], 6: [1, 1, 0], 7: [1, 1, 1]}

and I can have the mentioned mapping eg

>>> b[1]
[0, 0, 1]

However, in my case the n=8 and m=16 and storing such a dictionary ( 8**16 entries.) will take more than 100 Gb? I want to know if there is a lazy way of generating each index of the b everytime I want to access that without generating the entire dictionary?

Answer 1

You can generate the xth element by taking the successive x//n**(i-1)%n for i that varies between m-1 and 0.

n = 3
m = 4

x = 6
[x//n**(i-1)%n for i in range(m, 0, -1)]

output: [0, 0, 2, 0]

as a function

def f(x, n, m):
    return [x//n**(i-1)%n for i in range(m, 0, -1)]

f(1, n=2, m=3)
# [0, 0, 1]

f(1234567890, n=8, m=16)
# [0, 0, 0, 0, 0, 1, 1, 1, 4, 5, 4, 0, 1, 3, 2, 2]

Answer 2

You can make a function instead of full map. Your output is actually array of size m representing value of input a in n-th scale of notation. If you want to make a map just save the result of the function in map if key doesn't exist.

def f(a, n, m):
    res = []
    while a != 0:
        res = [a % n] + res
        a //= n
        m -= 1
    while m != 0:
        res = [0] + res
        m -= 1
    return res

Answer 3

If you want to manipulate the permutations as if you had a list but without generating the actual list, you can make a function that will give you then Nth permutation in a "virtual" list:

from math import factorial
def permCount(A,k): return factorial(len(A))//factorial(len(A)-k)
def permAtIndex(A,k,index):
    A      = list(A)
    base   = permCount(A,k)
    result = []
    for _ in range(k):                    # for required size
        base  //= len(A)                  # chunk size for position
        i,index = divmod(index,base or 1) # i is value index at position
        result.append(A.pop(i))           # build permutation
    return result 

This will give you the size of the permutation "list" and allow you to get a permutation at a given index:

print(permCount([1,2,3,4],4))       # 24
print(permAtIndex([1,2,3,4],4,22))  # [4, 3, 1, 2]

print(permCount(range(15),10))              # 10897286400
print(permAtIndex(range(15),10,5897286400)) # [8,1,10,5,12,0,13,7,14,9]

You can also create a reverse function to give you the index of a given permutation:

def indexOfPerm(A,k,P):
    A      = list(A)
    base   = permCount(A,k)
    result = 0
    for n in P:            
        base //= len(A)  # chunk size for position
        i = A.index(n)   # index in remaining items
        result += base*i # position's component of index
        A.pop(i)         # next position on rest of items
    return result

print(indexOfPerm([1,2,3,4],4,[4, 3, 1, 2])) # 22
print(indexOfPerm(range(15),10,[8,1,10,5,12,0,13,7,14,9])) # 5897286400