set directory for search program in python

Question

I am trying to develop a CNN for image processing. I have about 130 gigs stored on a separate drive on my comp, and I'm having trouble navigating a simple python search program to search through that specified directory. Im trying to have it find a bunch of random XML files scattered in a host of sub-directories/sub-directories/subs on that drive. How do I specify for just this one python program the directory it should be searching in, keeping it only to the context of the program?

Ive tried setting a variable Path = "B:\\\\MainFolder\\SubFolder" and using os.walk , but it makes it through the first directory then stops.

Answer 1

can you try the following:

import os
import glob
base_dir = 'your/start/sirectory'
req_files = glob.glob(os.path.join(base_dir, '**/*.xml'), recursive=True)

Answer 2

Jeril and Eduardo, thank you for the help. i took a shot at pathlib and it worked. idk what was up with my glob code, looked basically the same as yours Jeril:

import glob, os

filelist = []

from pathlib import Path

for path in Path('B:\\CTImageDataset\LIDC-IDRI').rglob('*.xml'):
    filelist.append(path.name)
    print(filelist)

Worked great, thanks again