Intersect a list of dicts based on a common key

Question

Let's say I have two list of dicts:

dates = [{'created':'2010-12-01'},{'created':'2010-12-02'},....]
elts = [{'created':'2010-12-01', 'key1':'val1', 'key2':'val2'}, {'created':'2010-12-05','key1':'val1'}]

The dates list is a bunch of contiguous dates.

The elts list can be anywhere from 1 to len(dates), and what I want to do is basically pad elts so that it has a dict for a date regardless if there're other keys.

This is my naive solution:

for d in dates:
    for e in elts:
        if d['created'] == e['created']:
            d.update(dict(key1=e['key1']))

Thus I will have a final array d with all dates in each dict , but there may/may not be other key/vals.

What's a good 'pythonic' solution?

Answer 1

Your question is a little off, I think, since your solution doesn't seem to actually address your question, but if you wanted to create an entry in elts for every date in dates that does not already appear in elts , you could use this:

all_dates = set(e['created'] for e in dates) # gets a list of all dates that exist in `dates`
elts_dates = set(e['created'] for e in elts) # same for elts

missing_dates = all_dates - elts_dates

for entry in missing_dates:
    elts.append(dict(created=entry))

Here's a http://codepad.org snippet that shows this snippet in effect: http://codepad.org/n4NbjvPM

Answer 2

EDIT: Different solution:

Make a set of dates you've already got:

dates_in_elts = set(e['created'] for e in elts)

for d in dates:
    if d['created'] not in dates_in_elts:
        e.append(d)

This only iterates over each list once, rather than iterating over elts for each date in dates.

Answer 3

I'd probably make those lists dictionaries instead.

  dates_d = dict([(x['created'], x) for x in dates])
  elts_d = dict([(x['created'], x) for x in elts])
  dates_d.update(elts_d)

If you then need it to be a list of dicts in order again, you can do that easily:

  dates = [dates_d[x] for x in sorted(dates_d)]

If you are not doing anything else than merging them, your solution might be more easily readable, though. But lists of dictionaries is not a very handy format for data in this case, I suspect.

Answer 4

Maybe I've misread, but it seems to me that the end result of your code is that, for every dict in elts, you really want to just copy that dict from elts to overwrite the corresponding dict in dates.

>>> for d in dates:
...    for e in elts:
...       if d['created'] == e['created']:
...          d.update(e)

At that point, it's the dates dictionary that reflects what I think you want.