“unpacking” a passed dictionary into the function's name space in Python?

Question

In the work I do, I often have parameters that I need to group into subsets for convenience:

d1 = {'x':1,'y':2}
d2 = {'a':3,'b':4}

I do this by passing in multiple dictionaries. Most of the time I use the passed dictionary directly, ie:

def f(d1,d2):
    for k in d1:
        blah( d1[k] )

In some functions I need to access the variables directly, and things become cumbersome; I really want those variables in the local name space. I want to be able to do something like:

def f(d1,d2)
    locals().update(d1)
    blah(x)
    blah(y)    

but the updates to the dictionary that locals() returns aren't guaranteed to actually update the namespace.

Here's the obvious manual way:

def f(d1,d2):
    x,y,a,b = d1['x'],d1['y'],d2['a'],d2['b']
    blah(x)
    return {'x':x,'y':y}, {'a':a,'b':b}

This results in three repetitions of the parameter list per function. This can be automated with a decorator:

def unpack_and_repack(f):
    def f_new(d1, d2):
        x,y,a,b = f(d1['x'],d1['y'],d2['a'],d3['b'])
        return {'x':x,'y':y}, {'a':a,'b':b}
    return f_new
@unpack
def f(x,y,a,b):
    blah(x)
    blah(y)
    return x,y,a,b

This results in three repetitions for the decorator, plus two per function, so it's better if you have a lot of functions.

Is there a better way? Maybe something using eval? Thanks!

Answer 1

You can always pass a dictionary as an argument to a function. For instance,

dict = {'a':1, 'b':2}
def myFunc(a=0, b=0, c=0):
    print(a,b,c)
myFunc(**dict)

Answer 2

If you like d.variable syntax better than d['variable'] , you can wrap the dictionary in an almost trivial "bunch" object such as this:

class Bunch:
    def __init__(self, **kw):
        self.__dict__.update(kw)

It doesn't exactly bring dictionary contents into the local namespace, but comes close if you use short names for the objects.

Answer 3

Assuming all keys in your dictionary qualify to be identifiers, You can simply do this:

adict = { 'x' : 'I am x', 'y' : ' I am y' }
for key in  adict.keys():
  exec(key + " = adict['" + key + "']")
blah(x)
blah(y)

Answer 4

This is similar to your decorator idea, but it is a bit more general, as it allows you to pass an arbitrary number of dicts to foo , and the decorator does not have to know anything about the keys in the dicts or the order of the arguments when calling the underlying foo function.

#!/usr/bin/env python
d1 = {'x':1,'y':2}
d2 = {'a':3,'b':4}

def unpack_dicts(f):
    def f_new(*dicts):
        new_dict={}
        for d in dicts:
            new_dict.update(d)
        return f(**new_dict)
    return f_new

@unpack_dicts
def foo(x,y,a,b):
    print x,y,a,b

foo(d1,d2)
# 1 2 3 4

Answer 5

This is what I use as a locals().update(d1) workaround:

def f(d1,d2)
    exec ','.join(d1) + ', = d1.values()'
    blah(x)
    blah(y)

Answer 6

Here's a unpacking method in a one liner:

x,y = (lambda a,b,**_: (a,b))(**{'a':'x', 'b':'y', 'c': 'z'})

the lambda args a and b are the keys I want to unpack into x and y in that order. **_ is there to ignore any other keys in the dictionary, ie c .

Answer 7

I do not think you can get any more convenience for dict unpacking in Python. So, here comes the obligatory "if that hurts, don't do that" answer.

Mapping item access IS more cumbersome than attribute access in Python, so maybe you should pass instances of user-defined classes instead of dicts.

Answer 8

I think the common wisdom is "don't use the inspect module in production code", and I mostly agree with that. As such, I think it is a bad idea to do the following in production code. But, if you're working on a python that supports frames (like CPython), this should work:

>>> def framelocals():
...    return inspect.currentframe(1).f_locals
... 
>>> def foo(ns):
...    framelocals().update(ns)
...    print locals()
... 
>>> foo({'bar': 17})
{'ns': {'bar': 17}, 'bar': 17}

It just grabs the actual dict out of the caller's frame, which when called inside a function body, should be the function's name space. I don't know if there is or isn't a situation when using CPython when locals() doesn't just do this anyway; the warning in the documentation might be to say "the effects of modifying the dict returned by locals() are python implementation dependent". Thus, while it works to modify that dict in CPython, it may not in another implementation.

UPDATE: This method doesn't actually work.

>>> def flup(ns):
...    framelocals().update(ns)
...    print locals()
...    print bar
... 
>>> flup({'bar': 17})
{'ns': {'bar': 17}, 'bar': 17}
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 4, in flup
NameError: global name 'bar' is not defined

As ddaa suggested, there's some deeper magic in the function compilation which makes note of the local variables. Thus you can update the dict , but you can't see the update with normal local namespace lookup.

Answer 9

You can use sorcery :

from sorcery import unpack_dict

x, y = unpack_dict(d1)

Answer 10

I am quiet frequently using .items() method for this.

def unpack(dict):
    for key, value in dict.items():
        print(key, value)

Answer 11

I wrote a Python package called var_arguments for conveniently bundling and unbundling arguments that should be useful here; it's available on github .

Instead of writing, say:

def f(d1,d2):
    x,y,a,b = d1['x'],d1['y'],d2['a'],d2['b']
    y=x+a
    return {'x':x,'y':y}, {'a':a,'b':b}

You could write:

from var_arguments import recon_dict, use_dargs

def f(d1,d2):
  r=f2(dargs=[d1,d2])
  return recon_dict(d1,r), recon_dict(d2,r)

@use_dargs
def f2(x,y,a,b):
  y=x+a
  return locals()

I wrote the solution like this to match what you seem to be going for: the dictionaries arrive and leave in groups and we minimize the number of times we mention the names of the keys in the dictionaries and/or manually access them. Specifically, we only need to mention x,y,a, and b once this way.

How it works, basically, is that @use_dargs modifies f2 so that it accepts an optional dargs keyword argument, which, if present, should supply a list of dictionaries (dargs=[d1,d2]). The key/value pairs in those dictionaries are added to the keyword arguments that are otherwise supply to the call to the function, with the keyword arguments having highest priority, d2 having second highest, and d1 having lowest priority. Accordingly you can call f2 in various ways and get the same result:

f2(1,2,3,4)
f2(1,a=3,dargs=[dict(y=2,b=4)])
f2(dargs=[dict(x=1,y=2),dict(a=3,b=4)])

recon_dict is for the case when you have one dictionary that holds old values for all the keys that you're interested in, and another dictionary that holds the new values for all those keys (as well as possibly others that you don't want). For example:

old_d=dict(a=8,b=9) # I want these keys, but they're old values
xyab=dict(x=1,y=2,a=3,b=4) # These are the new values, but I don't want all of them
new_d=recon_dict(old_d,xyab)
assert new_d==dict(a=3,b=4)

Here are some additional tricks to remove the redundancy of mentioning variable names multiple times within a function body that var_arguments handles. First, we can change:

{'x':x,'y':y}

into:

ddict('x,y',locals())

Similary, we can change:

f(x=x,y=y)

into:

dcall(f,'x,y',locals())

More generally, if we have a dictionary xy with keys x and y and if our local variables include a and b, we can change:

f(x=xy['x'],y=xy['y'],a=a,b=b)

into:

ldcall(f,'x,y,a,b',[locals(),xy])