Pythonic way of searching a dict of lists?

Question

I like to use the 'dict-of-lists' structure to organize data in Python for various reasons, however, a drawback I've found is it can be annoying to perform lookups.

Take an example where the data is the history of NCAA tournament finals--it would be something like:

    data = {'year': ['2019','2018',...],'winner': ['Virginia','Villanova',...], 'score':[]...etc}

Now suppose I wanted to find data on all the years that UConn won, as of now the way I do that is:

    columns = ['year','winner','coach','score','runner_up'] 
    new_dict = {i: [] for i in columns}                      #redefine blank dict
    for i in range(0,len(old_dict['year']),1):               #go through range
        for j in columns:                                    #go through columns
            if old_dict['winner'][i]=='Connecticut':
                new_dict[j].append(old_dict[j][i])           #append chosen data to new dict

    ....new_dict = {'year': ['2014','2011',...],'winner': ['Connecticut','Connecticut',...]..etc}

Is there any way to do this better? It feels kind of cumbersome and ugly. I've tried stuff with list/dict comprehension but am struggling. Since this type of situation will come up rarely in the projects I do I'd prefer to use generic Python and am not sure something more apt for the job like Pandas/Numpy is worth it.

Thanks!

Answer 1

The code that you are writing is re-creating a relational database, badly.

If you're going to be accessing your data along one pattern only, use a nested data structure that supports that access pattern. For two access patterns, either transform your data structure or maintain two nested data structures with a wrapper and logic to keep them synchronized. By the time you get to 3 or more access patterns, use a relational database.

If you want to keep it lightweight, just db = sqlite3.connect("file::memory:?cache=shared") to create an in memory SQLite database, and then work with that.