Python 2.7 List of Lists Manipulation

Question

I am new to Python 2.7. Below is a subset of my data, which is just a list of lists, and would like to sum the values of the lists with both the same type ( myList[0] ) and same dates ( myList[8] ) into a new single list.

myList = [
    [u'MtM', u'Consistency Check', u'Break', 123123, 123, 456456, 456, '1/8/2014'],
    [u'MtM', u'Consistency Check', u'Break', 321321, 321, 654654, 654, '1/8/2014'],
    [u'MtM', u'Consistency Check', u'Break', 987987, 987, 789789, 789, '1/9/2014'],
    [u'Notional', u'Consistency Check', u'Break', 320320, 320, 210210, 210, '1/13/2014'],
    [u'Notional', u'Completeness Check', u'Break', 121121, 121, 412412, 412, '1/13/2014']
]

I am trying to get the result below but do not know how to do so:

newList = [
    [u'MtM', u'Consistency Check', u'Break', 444444, 444, 1111110, 1110, '1/8/2014'],
    [u'MtM', u'Consistency Check', u'Break', 987987, 987, 789789, 789, '1/9/2014'],
    [u'MtM', u'Consistency Check', u'Break', 320320, 320, 210210, 210, '1/13/2014'],
    [u'Notional', u'Consistency Check:Completeness Check', u'Break', 441441, 441, 622622, 622, '1/13/2014']
]

In the new list of lists ( newList ), the lists with the same dates of the same type were combined into a single list but with with their numerical values summed and the description item ( myList[1] ) values concatenated with a ":" .

Does anyone know how to do this? Any ideas or advice would be greatly appreciated. Thank you!

Answer 1

Use a dictionary to map entries to a common sum:

sums = {}
for entry in myList:
    key = entry[0], entry[-1]
    if key not in sums:
        sums[key] = entry[:]
        sums[key][1] = set([entry[1]])
    else:
        sums[key][3:-1] = [s + elem for s, elem in zip(sums[key][3:-1], entry[3:-1])]
        sums[key][1].add(entry[1])

newList = [e[:1] + [':'.join(e[1])] + e[2:] for e in sums.values()]

1. The items are grouped on the first and last elements.
2. All but the first 3 and the last column are summed.
3. Output order can be arbitrary.
4. The 3rd column ( u'Break' ) doesn't vary.

Demo:

>>> myList = [
...     [u'MtM', u'Consistency Check', u'Break', 123123, 123, 456456, 456, '1/8/2014'],
...     [u'MtM', u'Consistency Check', u'Break', 321321, 321, 654654, 654, '1/8/2014'],
...     [u'MtM', u'Consistency Check', u'Break', 987987, 987, 789789, 789, '1/9/2014'],
...     [u'Notional', u'Consistency Check', u'Break', 320320, 320, 210210, 210, '1/13/2014'],
...     [u'Notional', u'Completeness Check', u'Break', 121121, 121, 412412, 412, '1/13/2014']
... ]
>>> sums = {}
>>> for entry in myList:
...     key = entry[0], entry[-1]
...     if key not in sums:
...         sums[key] = entry[:]
...         sums[key][1] = set([entry[1]])
...     else:
...         sums[key][3:-1] = [s + elem for s, elem in zip(sums[key][3:-1], entry[3:-1])]
...         sums[key][1].add(entry[1])
... 
>>> [e[:1] + [':'.join(e[1])] + e[2:] for e in sums.values()]
[[u'MtM', u'Consistency Check', u'Break', 987987, 987, 789789, 789, '1/9/2014'],
 [u'Notional', u'Completeness Check:Consistency Check', u'Break', 441441, 441, 622622, 622, '1/13/2014'],
 [u'MtM', u'Consistency Check', u'Break', 444444, 444, 1111110, 1110, '1/8/2014']]