skip hidden files and directories in pathlib glob

Question

With

from pathlib import Path

path = Path("/my/path")
for p in path.rglob("*"):
    print(p)

I can get all files and directories in /my/path . How can I skip hidden files and directories?

From

file
.hidden_file
dir /
   file
   .hidden_file
.hidden_dir / 
   file
   .hidden_file

I just want to get

file
dir /
   file

Answer 1

Instead of: path.rglob('*')

Try this: path.rglob('[..]*') if not os.path.isdir(p)

Unfortunately I can't test now, but hopefully it'll do.

Answer 2

There is no builtin way to do this, and glob definitely in not suitable for it. You can build it on top of os.walk I guess: filter the files array at each step, as well as modify dirs in-place to avoid recursing into hidden directories.

Do note that the Unix conception of hidden files (the dotfile hack) isn't necessarily sufficient (or compatible, kind-of) for other OS eg macOS does use dotfiles but also has a stat flag ( UF_HIDDEN ), and Windows should only use the relevant stat flag ( FILE_ATTRIBUTE_HIDDEN ) though ignoring dotfiles probably doesn't help.

Ideally there'd be a filewalker which would do that automatically, including support for ignorefiles (similar to Rust's ignore library), but pypi doesn't really surface anything, half the packages are about creating ignorefiles, and the rest is either incomplete or abandoned.

Answer 3

Great Question, there are two ways that I can think of that might help you depending on your use case. using os or pathlib.

from os import scandir
from pathlib import Path
# did shorthand for directory in directory
did = Path('path/to/parent/directory')
# with pathlib.Path()
paths = [dir_.as_posix() for dir_ in did.iterdir() if dir_.name[0]!='.']
# with os.scandir()
paths_ = [dir_.path for dir_ in scandir(did) if dir_.name[0]!='.']
paths_==paths
>>>True

Answer 4

How about something like:

visible = filter(
    lambda path: not any([part for part in path.parts if part.startswith(".")]),
    Path("my_path").rglob("*")
)

Answer 5

I think the most simple option would be

from pathlib import Path

path = Path("/my/path")

[f for f in path.rglob('*') if not any([p for p in f.resolve().parts if p.startswith(".")])]

Note the resolve call to deal with relative paths as well.