我怎样才能加快这个目录走？

Question

I'm trying to write a function that finds a directory by traversing a directory tree with os.walk() below. On my machine, this takes 15 seconds.

for dir_path, dir_names, filenames in os.walk(os.path.expanduser('~')):
    for dir_name in dir_names:
        if dir_name == 'some_dir':
            path = os.path.join(dir_path, dir_name)
            print(path)

I read that os.scandir() is faster so I tried this below, though I think the implementation is wrong. It works but it's now nearly 30 seconds.

for dir_path, dir_names, filenames in os.walk(os.path.expanduser('~')):
    with os.scandir(dir_path) as entries:
        for entry in entries:
            if entry.name.endswith('some_dir') and entry.is_dir():
                print(entry.path)

How can I speed this up?

Answer 1

One suggestion would be to replace the for dir_name in dir_names part:

for dir_path, dir_names, filenames in os.walk(os.path.expanduser('~'):
    if 'some_dir' in dir_names:
        path = os.path.join(dir_path, 'some_dir')
        print(path)

I do not know if you have lots of subdirectories, but depending on that, this should already make the code faster.

Additionally, I would suggest to comment out th

The os.scandir() function is preferred over os.listdir , when you need additional information of the file type, but as dir_names only contains the directories of the subfolder, this is additional overhead you produce by calling that function, hence it is a lot slower than the original code.

If you use a python version >=3.5 , then os.walk() already calls os.scandir() under the hood. As Charles Duffy already mentioned, individually calling os.scandir() recursively will most likely not be much faster.