找不到文件 Jupyter 笔记本

Question

I am having trouble loading a file in jupyter notebook.

Here is my project tree:

-- home

---- cdsw

------ my_main.py

------ notebooks

-------- my_notebook.ipynb

------ dns

-------- assets

---------- stopwords.txt

-------- bilans

---------- my_module.py

Know that '/home/cdsw/" is in my PYTHONPATH - the same interpreter in which I launch jupyter -.

In my_module.py I have these lines:

PATH_STOPWORDS: Final = os.path.join("dns", "assets", "stopwords.txt")
STOPWORDS: Final = load_stopwords(PATH_STOPWORDS)

load_stopwords is basically just a open(PATH_STOPWORDS, 'r'). So my problem is that when I import dns.bilans.my_module inside my_main.py it works fine: file is correctly loaded. Yet, when I import it from my_notebook.ipynb, it does not:

FileNotFoundError: [Errno 2] No such file or directory: 'dns/assets/stopwords.txt'

So my_module is indeed founded by jupyter kernel (because it reads the code lines of the file) but can't use the relative path provided like it does from a run in a terminal.

When I use a open(relpath, 'r') inside a module, I don't need to go all through the project tree right? Indeed it DOES work in my_main.py...

I really don't get it...

The output of os.getcwd() in jupyter is "/home/cdsw/notebooks".

Answer 1

This existing SO question suggests how to find files relative to the position of a Python code file. It isn't exactly the same question, however, and I believe that this technique is so important for every Python programmer to understand, that I'm going to provide a more thorough answer.

Given a piece of Python code, one can compute the path of the directory of the source file containing that code via:

here = os.path.dirname(__file__)

Having the position of the relevant source file, it is easy to compute an absolute path to any data file that has a well known location relative to that source file. In this case, the way to do that is:

stopwords_path = os.path.join(here, '..', '..', 'assets', 'stopwords.txt')

This path can be supplied to open() or used in any other way to refer to the stopwords.txt data file. Here, the way to use this path would be:

load_stopwords(stopwords_path)

I use this technique to not only find files that accompany code in a particular module, but also to find files that are in other locations throughout my source tree. As long as the code and data file exist in the same source repository, or are shipped together in a single Python package, the relative path will not change from installation to installation, and so this technique will work.

In general, you should avoid the use of relative paths. Whenever possible, you should also avoid having to tell your code where to find something. For any situation, ask yourself how you can obtain a reliable absolute path that you can then use to then locate whatever it is you're wanting to access.