python join list of list of lists using key

Question

I have this list structure:

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]]]

'lst' can contain an arbitrary number of sublists (len(lst) can be bigger than 2)

As an output I want:

output = [['a',100,50],['b',200,250],['c',0,75],['d',325,0]]

Here is another example:

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b', 10]]]

output = [['a', 100, 50, 22],['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

How would you do that?

Answer 1

You can use a defaultdict :

from collections import defaultdict
import itertools
d = defaultdict(list)
lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]]]
for a, b in itertools.chain.from_iterable(lst):
   d[a].append(b)

new_lst = sorted([list(itertools.chain.from_iterable([[a], [0 for i in range(len(max(d.items(), key=lambda x:len(x[-1])))-len(b))]+b])) for a, b in d.items()])

Output:

[['a', 100, 50], ['b', 200, 250], ['c', 0, 75], ['d', 0, 325]]

Answer 2

This task would be a little simpler if we had a list of all the letter keys used in lst , but it's easy enough to extract them.

My strategy is to convert the sublists into dictionaries. That makes it easy & efficient to grab the value associated with each key. And the dict.get method allows us to supply a default value for missing keys.

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]]]

# Convert outer sublists to dictionaries
dicts = [*map(dict, lst)]

# Get all the keys
keys = set()
for d in dicts:
    keys.update(d.keys())

# Get data for each key from each dict, using 0 if a key is missing
final = [[k] + [d.get(k, 0) for d in dicts] for k in sorted(keys)]
print(final)

output

[['a', 100, 50], ['b', 200, 250], ['c', 0, 75], ['d', 325, 0]]

---

If we use

lst = [[['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b', 10]]]

then the output is

[['a', 100, 50, 22], ['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

---

If you want to run this on Python 2 you need to make a minor change to the code that converts the outer sublists to dictionaries. Change it to

dicts = list(map(dict, lst))

That will work correctly on both Python 2 & 3. And if you only need to run it on Python 2, you could simply do

dicts = map(dict, lst)

since map in Python 2 return a list, not an iterator.

Answer 3

With itertools.chain.from_iterable() , itertools.groupby() functions and built-in next() function:

import itertools

lst = [ [['a', 100],['b', 200],['d', 325]],[['a', 50],['b', 250],['c', 75]], [['a', 22], ['b', 10]] ]
lst_len = len(lst)
sub_keys = [{k[0] for k in _} for _ in lst]
result = [[k] + [next(g)[1] if k in sub_keys[i] else 0 for i in range(lst_len)]
          for k,g in itertools.groupby(sorted(itertools.chain.from_iterable(lst), key=lambda x:x[0]), key=lambda x: x[0])]

print(result)

The output:

[['a', 100, 50, 22], ['b', 200, 250, 10], ['c', 0, 75, 0], ['d', 325, 0, 0]]

Answer 4

This is my "long-hand" method, I just had to work out what was going on :

lst = [[['a', 100],['b', 200],['d', 325]],
      [['a', 50],['b', 250],['c', 75]],
      [['a', 22], ['b', 10]],
      [['c', 110],['f', 200],['g', 425]],
      [['a', 50],['f', 250],['h', 75]],
      [['a', 32], ['b', 10]], ]
nlist = []
store={}
for n,j in enumerate(lst):
    for i in j  :
        if i[0] in store :
            store[i[0]].append(i[1])
        else :
            store[i[0]] = nlist + [i[1]]
    nlist += [0]
    for k,v in store.items() :
        if len(v) < n+1 :
            store[k] = v + [0]
print(store)
result=[]
for k,v in store.items():
    result += [[k] + v]
print(sorted(result))