正则表达式和os.walk

Question

I'm pretty new to python but learning fast. I'm trying to use regex with os.walk to ignore directories that I don't want processed. I understand that you must modify the dirs in place and not create a new list. I have tried it both ways though. I don't get any errors but it still traverses all the directory's. Exluding full dir names works fine. I am trying to remove all directories with 'EXP' or '-' or '3.2' in the name. Here is an example I want to ignore 3.2.2.150-20150424.195805_EXP_manuMain_outOfMemFix

This is what I have:

def runtest(filepath_udu: object) -> object:   
    k = 1
    for root, dirs, files in os.walk(filepath_udu, topdown=True):
        dirs[:] = [item for item in dirs 
                   if item not in ('1node','local','remote')]
        dirs[:] = [dir for dir in dirs 
                   if re.search(r'\bEXP\b', dir) not in dirs \
                   or re.search(r'\b3.2\b', dir) not in dirs \
                   or re.search(r'\w+(?:- \w+)+', dir) not in dirs]
    for file in files:
        do something...

What am i doing wrong that my 2nd dirs[:] is being ignored? Thanks

Answer 1

Its not being ignored, it's just that your condition is always true, so you aren't filtering anything out.

re.search is going to return a match object if something is found, or None if not. Either way, that's not going to be an element of dirs , because dirs is just a list of strings. So all of your tests are always true.

Instead of checking that the search is not in dirs , just check that it's not truthy. (A match object is always truthy, and None is always falsey.)

Also, after you fix that, I'm pretty sure you wanted to keep the values where all of the tests fail—but you're using or instead of and , which means you're keeping the values where any of the tests fail.

So:

dirs[:] = [dir for dir in dirs
           if not re.search(r'\bEXP\b', dir)
           and not re.search(r'\b3.2\b', dir) 
           and not re.search(r'\w+(?:-\w+)+', dir)]

Or, if it's easier to understand the other way round—instead of keep all values where all the tests fail, keep all values where none of the tests is true:

dirs[:] = [dir for dir in dirs if not (
           re.search(r'\bEXP\b', dir) or
           re.search(r'\b3.2\b', dir) or
           re.search(r'\w+(?:-\w+)+', dir))]

Answer 2

Instead of using os.walk , you can avoid the overhead of dealing with list manipulations by recursively traversing the sub-directories on your own with os.scandir after excluding those that match your exclusion criteria:

def runtest(filepath_udu: object) -> object:
    for entry in os.scandir(filepath_udu):
        if entry.is_dir() and entry.name not in ('1node', 'local', 'remote') and not re.search(r'\bEXP\b', entry.name) and not re.search(r'\b3.2\b', entry.name) and not re.search(r'\w+(?:- \w+)+', entry.name):
            runtest(entry.path)
        else:
            do something ...