How to loop over sub-directories within a root directory using os.walk()?

Question

I have a root directory termed "2013" with some subdirectories as follows:

2013/:
     1/:
         ANAR/
     2/:
         ANAR/

what I intend to do is to use os.walk() to go through the innermost sub-directories (ie, ANAR) and create a similar folder within them if it does not exist already. Here is my code:

for root, dirs, files in os.walk ('./2013/'):
   for f in dirs:
     if not f.isdigit():
         path = os.path.join (root, f)
         fullpath = f'{path}{os.path.sep}{f}'
         if not os.path.isdir (fullpath):
             os.mkdir (fullpath)

but I encounter the following error (creates a bunch of ANAR dirs within./2013/2/ANAR/):

OSError: [Errno 36] File name too long: './2013/2/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/ANAR/...

any ideas around the reason?

Answer 1

os.walk is not what you need here. You have a fixed starting setup, and all the directories you want to make are all on the same level, so using a recursive method like that is far more complex than required... particularly because you really shouldn't modify the subdirectory structure while iterating over it.

You could do this with os.listdir :

import os

root = '2013'

for entry in os.listdir(root):
    path = os.path.join(root, entry)
    if os.path.isdir(path):
        os.mkdir(os.path.join(path, 'ANAR', 'ANAR'))

But you mentioned in your comment that the numbered folders are months. Will you always have all twelve of them? If so, you don't even need to iterate over the directory.

import os

for month in range(1, 13):
    os.mkdir(os.path.join('2013', str(month), 'ANAR', 'ANAR'))

In either event though, I'd recommend using pathlib over os , as it canmake your code a lot clearer . In particular, it handles the proper path separator for your OS, and you can just use forward slashes.

from pathlib import Path

for month in range(1, 13):
    Path(f'2013/{month}/ANAR/ANAR').mkdir()

Or, alternatively, if you might have an incomplete month list and want to iterate over the folders:

from pathlib import Path

for entry in Path('2013').iterdir():
    if entry.is_dir():
        (entry / 'ANAR/ANAR').mkdir()

(Note that the / operator can concatenate strings (or other Path objects) onto existing Pat objects.)

And finally, these will both throw an error if some of those ANAR subfolders already exist. If it's possible some might already be there (perhaps you started the script but had to abort it?), using Path.mkdir(exist_ok=True) will ignore such errors and keep going to create any that aren't there yet.