Python: maximum recursion depth

Question

A function for returning sum of the sizes of its arguments which could be single file/directory or a list of files/directories, is given below. The code gives an error message RuntimeError: maximum recursion depth exceeded while calling a Python object however I try to test this.

How to fix this?

Thanks

suresh

#!/usr/bin/python3.1
import os

def fileSizes(f):
    if hasattr(f,'__iter__'):
        return sum(filter(fileSizes,f))
    if os.path.isfile(f):
        return os.path.getsize(f)
    elif os.path.isdir(f):
        total_size = os.path.getsize(f)
        for item in os.listdir(f):
            total_size += fileSizes(os.path.join(f, item))
        return total_size

Answer 1

Instead of writing your own adhoc directory-transversal method, use the built-in os.walk (Documentation) method.

In fact, the example in the documentation (link above) computes the total size of non-directory files.

Answer 2

Last time I checked, the default maximum recursion depth was 1000. You have a few options:

1. Use Stackless Python which imposes no recursion limit (allowing infinite recursion).
2. Rewrite your function in an iterative style rather than recursive, so you don't overflow the stack in the first place.
3. Set the maximum recursion limit with sys.setrecursionlimit . Beware that this can cause a segmentation fault if abused.

Answer 3

There are many tricks to avoid problems with deep recursion, but I suspect that is not your core issue. (Maybe it is if you have very deep directory structures.. so test on a shallow directory first).

I'm guessing you're finding a cyclical structure in your directory tree caused by links. Ie a symlinked directory pointing back into a parent directory. (This kind of structure is not unusual).

You can avoid the infinite loop by checking whether a directory is a symlink before following it.

Another posibility is that you're getting "." or ".." somewhere in your lists. These will also generate infinite loops. I don't see that happening here, but its worth thinking about. os.listdir doesn't seem to return them.

Answer 4

The problem is in the line:

if hasattr(f,'__iter__'):
    return sum(filter(fileSizes,f))

Since f is a path, it is a string, and it has the attribute __iter__ , so you loop there infinitely.