Python open filename from custom PATH

Question

Similar to the system path, I want to offer some convenience in my code allowing a user to specify a file name that could be in one of a handful of paths.

Say I had two or more config paths

['~/.foo-config/', '/usr/local/myapp/foo-config/']

And my user wants to open bar , (AKA bar.baz )

1. Is there a convenient build in way to let open('bar') or open('bar.baz') automatically search these paths for that file in LTR order of precedence? Eg, will temporary adjusting my sys.path to only be these directories do this for me?

2. Else, how would you suggest implementing a PATH-like searching open-wrapper?

Answer 1

As other people already mentioned: sys.path only affects the module search path, ie it's relevant for importing Python modules, but not at all for open() .

I would suggest separating the logic for searching the paths in order of precedence and opening the file, because that way it's easier to test and read .

I would do something like this:

import os

PATHS = ['~/.foo-config/', '/usr/local/myapp/foo-config/']


def find_first(filename, paths):
    for directory in paths:
        full_path = os.path.join(directory, filename)
        if os.path.isfile(full_path):
            return full_path


def main():
    filename = 'file.txt'
    path = find_first(filename, PATHS)
    if path:
        with open(path) as f:
            print f
    else:
        print "File {} not found in any of the directories".format(filename)


if __name__ == '__main__':
    main()

Answer 2

open doesn't get into that kind of logic. If you want, write a wrapper function that uses os.path.join to join each member of sys.path to the parameter filename, and tries to open them in order, handling the error that occurs when no such file is found.

I'll add that, as another user stated, this is kind of a misuse of sys.path , but this function would work for any list of paths. Indeed, maybe the nicest option is to use the environment variables suggested by another user to specify a colon-delimited list of config directories, which you then parse and use within your search function.

Answer 3

environmental variables

say your app is named foo ... in the readme tell the user to use the FOO_PATH environmental variable to specify the extra paths

then inside your app do something like

 for path in os.environ.get("FOO_PATH",".").split(";"):
     lookfor(os.path.join(path,"somefile.txt"))

you could wrap it into a generic function

def open_foo(fname):
   for path in os.environ.get("FOO_PATH",".").split(";"):
       path_to_test = os.path.join(path,"somefile.txt")
       if os.path.exists(path_to_test):
              return open(path_to_test)
   raise Exception("No File Found On FOOPATH")

then you could use it just like normal open

with open_foo("my_config.txt") as f:
     print f.read()

Answer 4

Extract from Python Standard Library documentation for open built-in function :

open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)

...file is either a string or bytes object giving the pathname (absolute or relative to the current working directory ) of the file to be opened ...

Explicitely, open does not bring anything to automagically find a file : if path is not absolute, it is only searched in current directory.

So you will have to use a custom function or a custom class for that. For example:

class path_opener(object):
    def __init__(self, path = [.]):
        self.path = path
    def set(self, path):
        self.path = path
    def append(self, path):
        self.path.append(path)
    def extent(self, path):
        self.path.extend(path)
    def find(self, file):
        for folder in self.path:
            path = os.path.join(folder, file)
            if os.path.isfile(path):
                return path
        raise FileNotFoundError()
    def open(self, file, *args, **kwargs):
        return open(self.find(file), *args, **kwargs)

That means that a file opener will keep its own path , will be initialized by default with current path, will have methods to set, append to or extend its path, and will normaly raise a FileNotFoundError is a file is not found in any of the directories listed in its path.

Usage :

o = path_opener(['~/.foo-config/', '/usr/local/myapp/foo-config/'])
with o.open('foo') as fd:
    ...