How can I calculate all combinations of N different dictionaries in Python

Question

The Challenge

I need to combine various dictionaries together to obtain all possible combinations, whilst also having an approach that can take (N) number of dictionaries. I have the following code which works for a single case, but I'm seeking a more elegant and scalable solution.

Current Solution

This works for a specified number of dictionaries (profile, env, test_vars):

profile = {
    "normal": {"profile": "normal"},
    "special": {"profile": "special"}
}

environment = {
    "legacy": {"id": 123},
    "staging": {"id": 123, "sid": 123123},
    "production": {"pid": 14941729}
}

test_vars = {
    "no_tests": {"var1": ""},
    "var1:prod": {"var1": "prod"},
    "var1+var2": {"var1": "alpha", "var2": "alpha"},
    "var2_beta": {"var2": "beta"},
    "var2_alpha": {"var2": "alpha"}
}

total = [
    {"{}|{}|{}".format(pk, ek, tk): {**pv, **ev, **tv}}
    for pk, pv in profile.items()
    for ek, ev in environment.items()
    for tk, tv in test_vars.items()
]

result = [{id: '&'.join(["{}={}".format(i, x) for i, x in v.items()])
  for id, v in condition.items()}
 for condition in total]

result

Result :

The output looks like this, where each element of the list specifies a "test identifier" and the "test variables" expressed as querystring parameters.

Out[68]: 
[{'normal|legacy|no_tests': 'profile=normal&id=123&var1='},
 {'normal|legacy|var1:prod': 'profile=normal&id=123&var1=prod'},
 {'normal|legacy|var1+var2': 'profile=normal&id=123&var1=alpha&var2=alpha'},
 {'normal|legacy|var2_beta': 'profile=normal&id=123&var2=beta'},
 {'normal|legacy|var2_alpha': 'profile=normal&id=123&var2=alpha'},
 {'normal|staging|no_tests': 'profile=normal&id=123&sid=123123&var1='},
 {'normal|staging|var1:prod': 'profile=normal&id=123&sid=123123&var1=prod'},
 {'normal|staging|var1+var2': 'profile=normal&id=123&sid=123123&var1=alpha&var2=alpha'},
 {'normal|staging|var2_beta': 'profile=normal&id=123&sid=123123&var2=beta'},
 {'normal|staging|var2_alpha': 'profile=normal&id=123&sid=123123&var2=alpha'},
 {'normal|production|no_tests': 'profile=normal&pid=14941729&var1='},
 {'normal|production|var1:prod': 'profile=normal&pid=14941729&var1=prod'},
 {'normal|production|var1+var2': 'profile=normal&pid=14941729&var1=alpha&var2=alpha'},
 {'normal|production|var2_beta': 'profile=normal&pid=14941729&var2=beta'},
 {'normal|production|var2_alpha': 'profile=normal&pid=14941729&var2=alpha'},...

My issue

This isn't scalable to adding several more configuration dictionaries.

Let's say I now wanted to add another configuration dictionary, say, release . I'd have to add that new dict, plus amend the "total" comprehension like thus:

total = [
    {"{}|{}|{}|{}".format(pk, ek, tk, rk): {**pv, **ev, **tv, **rv}}
    for pk, pv in profile.items()
    for ek, ev in environment.items()
    for tk, tv in test_vars.items()
    for rk, rv in release_vars.items()
]

Which seems inelegant, especially when I could be dealing with 00's of configuration dictionaries.

Can anyone help devise something more scalable?

Answer 1

you could try something like this:

from itertools import product
def t(*args):
    counter = len(args)
    tmp  = [i.keys() for i in args]

    s =  {"|".join(i): [l[k] for l,k in zip(args,i)] for i in product(*tmp)}

    for k, v in s.items():
         s[k] = "&".join(["%s=%s"%(v1,v2) for j in v for v1,v2 in j.items()])
    return s

from pprint import pprint
pprint(t(profile, environment, test_vars))

Result:

{'normal|legacy|no_tests': 'profile=normal&id=123&var1=',
 'normal|legacy|var1+var2': 'profile=normal&id=123&var1=alpha&var2=alpha',
 'normal|legacy|var1:prod': 'profile=normal&id=123&var1=prod',
 'normal|legacy|var2_alpha': 'profile=normal&id=123&var2=alpha', ...

PS This is not necessary the best solution, but it could be scaled, good luck:)