How can I open files in directories below the working directory in Jupyter with Python on Linux?

Question

I am an intermediate Python and Jupyter developer, but I am having an issue with what should seem a simple problem.

Problem

When I open a notebook and call the rsf package command to input a file, I cannot read rsf files in a given directory below the working directory. All files are not empty.

with rsf.input(fname) as sf:

The error is: FileNotFoundError returned by the following line:

with(open(filename, 'r') as fh:

Attempted Solutions

1) I have used sys.path.append('/ <path from Jupyter working directory to file directory> / ), but still cannot read the files in that directory.

2) I have deleted the above sys.path.append(...) command and called sys.path.pop() to remove the file directory, and still have the same error.

3) I printed the sys.path to confirm and see that only python36 related directories are on the path, no custom directories I have added. I checked this after step 2.

4) I listed all files in the directory I specified, and I can see all expected files:

from os import listdir
from os.path import isfile, join
mypath = <path from Jupyter working directory to file directory>
onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]
print(onlyfiles)

Is there another working directory path that I need to load?

Answer 1

Try using os.listdir to see what files are visible to Python.

Also be aware that open doesn't use sys.path :

file is a path-like object giving the pathname (absolute or relative to the current working directory)

You can change the 'current working directory' using os.chdir .

You can see the current directory using os.path.abspath('.') .

Answer 2

Adding some more context here for sys.path vs os.path.

sys.path is used to change the path that Python searches to import packages: https://leemendelowitz.github.io/blog/how-does-python-find-packages.html

os.path is used to change the current working directory: https://www.geeksforgeeks.org/python-os-chdir-method/