Python: check for existence of a non-directory file at a given path

Question

I have a directory with a lot of files (~1.4M) from a data-dump and I want to sort them into subdirectories to create batches of smaller size. Due to the large number of files in the directory I want to batchify, I want to avoid having to os.listdir(...) everything in the directory and then test all of the results if they are a directory or not.

My idea until now is to use os.walk(...) to check this.

Is there a more pythonic/efficient way to achieve what I want?

Answer 1

As an alternative to os.listdir() , you can use os.scandir() on Python 3.5+. This returns an iterator of DirEntry objects and does not recurse into sub-directories as is the case with os.walk .

Here's an example from the docs :

for entry in os.scandir(path):
   if not entry.name.startswith('.') and entry.is_file():
       print(entry.name)