How can I get the keys from a nested dictionary?

Question

I have a list within a dictionary within a dictionary:

{FirmA:{ProductA:[Color1,Color2,Color3]}}

I want to build a list of keys from the First Firm dictionary level.

Then, I need to access the second level Product dictionary based on a Firm Key.

Finally, I will need to access the Colors list based on the Product key from Dictionary level 2 (Products).

I tried to get the level 1 keys for Firms:

[i for i in dict.keys()]

Returns

ValueError: Too many values to unpack

This is a fairly large data set.

I have not been able to get to the 2nd level dictionary yet.

Answer 1

What about that:

d = {'foo':{'bar':42}}

# you can do much like for nested list, like so:
print(d['foo'])
print(d['foo']['bar'])

# or you can iterate:
for k,v in d.items():
    print(k,v)

    # if the value is also a dictionnary, iterate on it again:
    try:
        for k2, v2 in v.items():
            if isinstance(v2, list):
                for el in v2:
                    print el
    except:
        pass

EDIT: actually if it's a large dataset, and you'll have few values below the first pass actually a dictionnary, it may be faster to do an instance check as well ( isinstance(v, dict)), since catching is expensive. Depends on the details....

Answer 2

Something like this should get you started:

def get_products_for_firm(d, firm):
    firm_letter = firm[-1]
    product_prefix = "Product"
    product_key = "%s%s" % (product_prefix, firm_letter)
    return d[firm][product_key]


d = {
    "FirmA": {
        "ProductA": ["Color1", "Color2", "Color3"]
    },
    "FirmB": {
        "ProductB": ["Color4", "Color5", "Color6"]
    }
}

firm_keys = d.keys()
print "The firm keys are %s: " % (firm_keys)

for firm_key in firm_keys:
    print "The products for %s are: %s" % (firm_key,
                                           get_products_for_firm(d, firm_key))

Output:

The firm keys are ['FirmA', 'FirmB']:
The products for FirmA are: ['Color1', 'Color2', 'Color3']
The products for FirmB are: ['Color4', 'Color5', 'Color6']

Answer 3

With a NestedDict you can extract keys of a nested dictionary as it was flat.

from ndicts import NestedDict

d = {"FirmA": {"ProductA": ["red", "blue", "yellow"],
               "ProductB": ["grey", "white"]},
     "FirmB": {"ProductB": ["black"]}}
nd = NestedDict(d)

>>> keys = list(nd.keys())
>>> keys
[('FirmA', 'ProductA'), ('FirmA', 'ProductB'), ('FirmB', 'ProductB')]

If you want to access a specific level use a comprehension

>>> 1st_level = [key[0] for key in keys]
>>> 2nd_level = [key[1] for key in keys]
>>> 1st_level, 2nd_level
(['FirmA', 'FirmA', 'FirmB'], ['ProductA', 'ProductB', 'ProductB'])

If you need to access an item:

>>> nd["FirmA", "ProductB"]
['grey', 'white']

To install ndicts pip install ndicts .

Finally, you said that you have a large dataset. While a NestedDict will give you what you want, it is not meant to be performing fast. It is simply meant to give you a simple interface to work with on nested dictionaries. Check out pandas Dataframes to speed up lookups. In particular have a look at MultiIndex, which are great to store hyerarchical data like yours.