Python - 按索引在列表列表中复制组

Question

I've seen a lot of questions about removing duplicates from a list and counting them. But I'm trying to find the best way to group them - for a list of lists.

Given this example I want to group by the third field:

[[1, "text", "name1", "text"],
 [2, "text", "name2", "text"],
 [3, "text", "name2", "text"],
 [4, "text", "name1", "text"]]

I'd like to get this:

[[[1, "text", "name1", "text"],
  [4, "text", "name1", "text"]],
 [[2, "text", "name2", "text"],
  [3, "text", "name2", "text"]]]

I can think of the naive way by looping through and just keeping track of what is found (O(n^2)). But I would assume there's a better way.

Answer 1

You could sorted and use groupby but that is O(n log n) :

from operator import itemgetter
from itertools import groupby

print([list(v) for _,v in groupby( sorted(l,key=itemgetter(2)),itemgetter(2))])

Or use an OrderedDict grouping by the third element for an O(n) solution by using the third element as the key and appending the sublists as values. setdefault will handle the repeated keys:

from collections import OrderedDict

od = OrderedDict()

for sub in l:
    od.setdefault(sub[2],[]).append(sub)
from pprint import pprint as pp
pp(od.values())
[[[1, 'text', 'name1', 'text'], [4, 'text', 'name1', 'text']],
[[2, 'text', 'name2', 'text'], [3, 'text', 'name2', 'text']]]

If order does not matter you can use a defaultdict in place of the OrderedDict.

If order does not matter a defaultdict is by far the most efficient.

In [7]: from itertools import groupby

In [8]: from collections import OrderedDict, defaultdict                               

In [9]: l = [[1, "text", "name{}".format(choice(list(range(2000)))), "text"] for _ in xrange(40000)]

In [13]: from operator import  itemgetter

In [14]: timeit [list(v) for _,v in groupby( sorted(l,key=itemgetter(2)),itemgetter(2))]
10 loops, best of 3: 42.5 ms per loop

In [15]: %%timeit                                                                       
od = defaultdict(list)
for sub in l:
    od[sub[2]].append(sub)
   ....: 
100 loops, best of 3: 9.42 ms per loop

In [16]: %%timeit                                                                       
od = OrderedDict()
for sub in l:
     od.setdefault(sub[2],[]).append(sub)
   ....: 
10 loops, best of 3: 25.5 ms per loop

In [17]: lists = l

In [18]: %%timeit
   ....: groupers = set(l[2] for l in lists)
   ....: [filter(lambda x: x[2] == y, lists) for y in groupers]
   ....: 

1 loops, best of 3: 8.48 s per loop

In [19]: timeit l = [filter(lambda x: x[2] == y, lists) for y in   set(l[2] for l in lists)]
1 loops, best of 3: 8.29 s per loop

So if order does not matter then defaultdict wins, groupby still performs pretty well as sort is still pretty cheap in comparison to a quadratic approach. As you can see filter's quadratic complexity performs badly as the data grows.

Answer 2

Here you go:

>>> lists = [[1, "text", "name1", "text"],
...  [2, "text", "name2", "text"],
...  [3, "text", "name2", "text"],
...  [4, "text", "name1", "text"]]
>>> groupers = set(l[2] for l in lists)
>>> groupers
set(['name2', 'name1'])
>>> l = [filter(lambda x: x[2] == y, lists) for y in groupers]
>>> pprint.pprint(l)
[[[2, 'text', 'name2', 'text'], [3, 'text', 'name2', 'text']],
 [[1, 'text', 'name1', 'text'], [4, 'text', 'name1', 'text']]]

You can of course write the whole grouping logic in a single line:

>>> l = [filter(lambda x: x[2] == y, lists) for y in set(l[2] for l in lists)]
>>> pprint.pprint(l)
[[[2, 'text', 'name2', 'text'], [3, 'text', 'name2', 'text']],
 [[1, 'text', 'name1', 'text'], [4, 'text', 'name1', 'text']]]

Answer 3

The easiest way of doing that is with the key argument of the sorted() function. In your example:

>>> a = [[1, "text", "name1", "text"], [2, "text", "name2", "text"], [3, "text", "name2", "text"], [4, "text", "name1", "text"]]

>>> sorted(a[:], key=lambda item:item[2])

>>> [[1, 'text', 'name1', 'text'], [4, 'text', 'name1', 'text'], [2, 'text', 'name2', 'text'], [3, 'text', 'name2', 'text']]

You can find more information about this argument on this link .

Answer 4

Use sorted with element you want to sort on as key and itertools groupby to group 'em:

>>> from itertools import groupby
>>> sl = sorted(your_list, key=lambda your_list: your_list[2])
>>> [list(v) for k,v in groupby(sl, key=lambda sl:sl[2])]
[[[1, 'text', 'name1', 'text'], 
  [4, 'text', 'name1', 'text']], 
 [[2, 'text', 'name2', 'text'], 
  [3, 'text', 'name2', 'text']]]

Answer 5

The following function will quickly ( no sorting required) group sub-sequences of any length by a key of specified index :

# given a sequence of sequences like [(3,'c',6),(7,'a',2),(88,'c',4),(45,'a',0)],
# returns a dict grouping sequences by idx-th element - with idx=1 we have:
# if merge is True {'c':(3,6,88,4),     'a':(7,2,45,0)}
# if merge is False {'c':((3,6),(88,4)), 'a':((7,2),(45,0))}
def group_by_idx(seqs,idx=0,merge=True):
    d = dict()
    for seq in seqs:
        if isinstance(seq,tuple): seq_kind = tuple
        if isinstance(seq,list): seq_kind = list
        k = seq[idx]
        v = d.get(k,seq_kind()) + (seq[:idx]+seq[idx+1:] if merge else seq_kind((seq[:idx]+seq[idx+1:],)))
        d.update({k:v})
    return d

In the case of your question, the key is the element having index 2, therefore

group_by_idx(your_list,2,False)

gives

{'name1': [[1, 'text', 'text'], [4, 'text', 'text']],
 'name2': [[2, 'text', 'text'], [3, 'text', 'text']]}

which is not exactly the output you asked for, but might as well suit your needs.