How can I return key value dictionary pairs from a list of dictionaries of unknown amount?

Question

I have data that is either being returned as a single dictionary, example:

{'key': 'RIDE', '3': 27.3531}

or as a list of dictionaries of unknown amount (ie. could be up to 20 dictionary lists or 2 as shown), example:

[{'key': 'GH', '3': 154.24}, {'key': 'RIDE', '3': 27.34}]

I'd like to write a piece of code that will iterate through the list of dictionaries and return all the key value pairs within each dictionary.

Any help would be appreciated, thank you!

Answer 1

To experiment with this we first have to write some code with a dummy data provider that either returns a dictionary or a list of dictionaries:

import random

def doSomething():
    if random.random() <= 0.5:
        return {'key': 'RIDE', '3': 27.3531}
    else:
        return [{'key': 'GH', '3': 154.24}, {'key': 'RIDE', '3': 27.34}]
#

Now we encounter exact this situation you described: That you sometimes receive a dictionary, sometimes a list of dictionaries.

Now: How to deal with this situation? It's not too difficult:

x = doSomething()
if isinstance(x, dict):
    x = [ x ]

for item in x:
    print(item)

So as we know we either receive a single dictionary or alternatively a list of dictionaries we can test of which type the value returned is. As it is much more convenient to always process a list the above example first converts the returned dictionary into a list before any additional data processing takes place.

So this is done by "correcting an error": The error is here that your data provider does not always return data of the same type. So whenever we receive a dictionary we first pack the dictionary into a list. Only after then we begin with the data processing we want to in the first place. Here this processing is to just print the dictionary, but of course you can iterate through the keys and values as well as you mentioned or do any kind of data processing you might feel appropriate.

But: It is not a good idea to have some kind of function or subsystem of any kind that returns different types of data. As you see this enforces to implement extra logic on the caller's side. Additionally it complicates things if we want to write a specification (and we WANT to write a specification in/for more complex programs.)

Example:

import typing
import random

def doSomething() -> typing.Union[dict,typing.List[dict]]:
    if random.random() <= 0.5:
        return {'key': 'RIDE', '3': 27.3531}
    else:
        return [{'key': 'GH', '3': 154.24}, {'key': 'RIDE', '3': 27.34}]
#

Here the specification is a formal one using the capabilities of typing . So this information about the returned type is specified in a formal way. Though this information is typically not evaluated by Python directly under normal circumstances this specification provides any programmer with the knowledge about returned types.

Of course we could have written such a specification in a non-formal way by writing some kind of text document as well, but that does not make any difference: In general having different types of return values complicates things unnecessarily. You can do that - there are situations where this makes sense - but in general it's better to avoid that situation as best as you can to simplify things.

For example using this approach:

import random

def doSomething() -> typing.List[dict]:
    # the old code we might choose not to change for some reason ...
    if random.random() <= 0.5:
        x = {'key': 'RIDE', '3': 27.3531}
    else:
        x = [{'key': 'GH', '3': 154.24}, {'key': 'RIDE', '3': 27.34}]

    # but we can compensate ...
    if isinstance(x, dict):
        x = [ x ]

    return x
#

Now we made the function to always return data of the same type. Which is now much more convenient for us: As a) it simplifies processing for the caller and b) simplifies learning about the data returned in the first place.

So having converted everything to return only data of a single type our main routine will simplify to this:

for item in x:
    print(item)

Or if you want to display keys and values:

for item in x:
    for k, v in item.items():
        print(k, "->", v)

Or whatever kind of data processing you have in mind with the data returned.

Remember as a rule of thumb, in any kind of scripting or programming language:

Always provide data in a way that it is easy for the caller to use and that the whole logic is easy to understand for the programmer. Make providing data in good a way a problem for the subroutine, not the caller. Simplify the caller's life as much as possible.

(Yes, you can decide to violate this principle and not follow it but if you do that then you really must have a very good reason. Then you really need to know what you're doing as then you have a very very special situation. Let me tell you from my 25 years of experience as a professional software developer: In 99.999% of all cases you will not have such a special situation. And I have the feeling that your situation does not fall into this category of such a special situation;-) )