How do I flatten nested lists of lists in python?

Question

I have created a function to split paths into lists of directories in python like so:

splitAllPaths = lambda path: flatten([[splitAllPaths(start), end] if start else end for (start, end) in [os.path.split(path)]])

with this helper function:

#these only work one directory deep
def flatten(list_of_lists):
    return list(itertools.chain.from_iterable(list_of_lists))

The output from this function looks like so:

> splitAllPaths('./dirname/dirname2/foo.bar')
[[[['.'], 'dirname'], 'dirname2'], 'foo.bar']

now I want this as a flat list. my attempts are as follows (with the output):

> flatten(splitAllPaths('./diname/dirname2/foo.bar'))
['.', 'd', 'i', 'r', 'n', 'a', 'm', 'e', 'd', 'i', 'r', 'n', 'a', 'm', 'e', '2', 'f', 'o', 'o', '.', 'b', 'a', 'r']

and

> reduce(list.__add__, (list(mi) for mi in splitAllPaths('./dirname/dirname2/foo.bar')))
me2/foo.bar')))
[[['.'], 'dirname'], 'dirname2', 'f', 'o', 'o', '.', 'b', 'a', 'r']

How do I unfold this list correctly (I would also welcome any suggestions for how to improve my splitAllPaths function)?

Answer 1

This a less general answer, but it solves your original problem -- although its elegance is debatable.

The main idea is the fact that generating a list with the reversed (as in ['file', 'user', 'home', '/'] order is quite easy, so you can just create that and reverse it in the end. So it boils down to:

def split_paths(path):                                                       
    def split_paths_reverse(path):                                           
        head, tail = os.path.split(path)                                     
        while head and tail:                                                 
            yield tail                                                                                
            head, tail = os.path.split(head)                                 
        yield head                                                           

    return reversed(tuple(split_paths_reverse(path)))

Example:

test = '/home/user/file.txt'
print(list(split_paths(test)))

['/', 'home', 'user', 'file.txt']

You could also avoid the explicit reversing part by putting each element in a stack and then removing them, but that's up to you.

Answer 2

Sortherst way that comes in mind would be:

listoflists = [[[['.'], 'dirname'], 'dirname2'], 'foo.bar']
str(listoflists).translate(None,"[]'").split(',')

Answer 3

I solved this by writing a (non-general) foldr. I think better, more practical solutions are provided by @L3viathan in the comments.

attempt = lambda list: attempt(list[0] + list[1:]) if len(list[0]) > 1 else list[0] + list[1:]

Output

> attempt([[[['.'], 'dirname'], 'dirname2'], 'foo.bar'])
['.', 'dirname', 'dirname2', 'foo.bar']

I've also now written it in terms of a general foldr1

> foldr1 = lambda func, list: foldr1(func, func(list[0], list[1:])) if len(list[0]) > 1 else func(list[0], list[1:])
> foldr1(list.__add__, [[[['.'], 'dirname'], 'dirname2'], 'foo.bar'])
['.', 'dirname', 'dirname2', 'foo.bar']

NOTE: Could someone more familiar than me confirm that this is a foldr and not a foldl (I often get them confused).