使用递归函数将嵌套列表转换为集合

Question

I need to create a recursive function (for simplicity) that takes any nested list and returns a set of unique elements.

To tackle this, I decided to first create a function that takes a list and converts it to a set:

ranList = [2, 2, 4, 5, 3, 1, 3]

def eue(ranList):
    newList = set(ranList)
    return print(newList)

Easy enough, it works. Now to create a function that takes a nested list and returns a 2-D list (this is a recursive function I found using search on this site) and a function that takes the other function, and returns a set containing unique elements:

lis = [['c', 'd'], 2, 2, 4, 5, 3, 1, 3, ['c']]

from collections import Iterable

def flatten(lis):
    for item in lis:
        if isinstance(item, Iterable) and not isinstance(item, str):
            for x in flatten(item):
                yield x
        else:        
            yield item

def eue(lis):
    newSet = set(flatten(lis))
    print(newSet)

Now, by calling eue(), it solves my original question. But I want to make it much more simplistic.

How would I go about combining these functions to produce a single function that reduces the amount of computing time it takes to run?

Thanks.

Answer 1

you can use itertools.chain to link all the iterators together:

from itertools import chain
from collections import Iterable

def isIter(obj):
    return isinstance(obj, Iterable) and not isinstance(obj,str)
def flatten(it):
    seqs = (flatten(item) if isIter(item) else (item,) for item in it)
    return chain(*seqs)
def enu1(it):
    return set(gen(it))

Although this means that for every non-sequence element you have to make a tuple with that one item (item,) for chain to work properly, not sure how much of an effect it would have on performance.

You can reduce this to a single function (other then isIter ) by converting to a set:

def enu2_A(it):
    seqs = (enu2_A(item) if isIter(item) else (item,) for item in it)
    return set(chain(*seqs))

But again this creates a new set object for every recursive call to enu , maybe add an option to convert?

def enu2(it,return_set = True):
    seqs = (enu2(item,False) if isIter(item) else (item,) for item in it)
    if return_set:
        return set(chain(*seqs))
    else:
        return chain(*seqs)

but combining it into a single function really doesn't give much speed boost at all:

import timeit

a = timeit.timeit("enu1(lis)","from __main__ import enu1,lis",number=10000)
b = timeit.timeit("enu2(lis)","from __main__ import enu2,lis",number=10000)

print(a,b)
print(a/b) #ratio, more then 1 means a took longer

output:

0.3400449920009123 0.32908301999850664
1.0333106582115827

so 3% faster by combining into a single function, I'm guessing that wasn't the speed up you were expecting, your code is very efficient as it is and much more pythonesque then mine so I wouldn't change it.

---

EDIT: just did a benchmark of my enu2 and your enu - your method is faster then the one I have provided by about 16% , leave it as it is as you can't get much better other then moving to python2 and using compiler.ast.flatten or another C level equivelent:

from collections import Iterable

def flatten(lis):
    for item in lis:
        if isinstance(item, Iterable) and not isinstance(item, str):
            for x in flatten(item):
                yield x
        else:        
            yield item

def enu(obj):
    return set(flatten(obj))

lis = [['c', 'd'], 2, 2, 4, 5, 3, 1, 3, ['c']]

import timeit

a = timeit.timeit("enu(lis)","from __main__ import enu,lis",number=10000)

b = timeit.timeit("set(ast.flatten(lis))","from __main__ import lis ; from compiler import ast",number=10000)

print(a,b)
print(a/b)

output:

0.3324121500045294 0.28561264199379366
1.1638565705076622

so doing the operation in C sped up the process by 4X, however the compiler package has been out of date since 2.6 according to the docs :

Deprecated since version 2.6: The compiler package has been removed in Python 3.

so it is quite possible that if you wrote flatten in a C extension you could get even better results but if you want to write code in pure python you have about as good as you can get.