Import custom modules in Amazon Sagemaker Jupyter notebook

Question

I want to import a custom module in my jupyter notebook in Sagemaker. Trying the import from Untitled1.ipynb I have tried two different structures. The first one is:

Inside "package folder" there were the files "cross_validation.py" and " init .py". The followings commands have been tried:

from package import cross_validation
import package.cross_validation

The second one is

and I have coded import cross_validation

In both cases I get no error at all when importing, but I can't use the class inside the module because I receive the error name Class_X is not defined

I also have restarted the notebook, just in case and it still not working. How could I make it?

Answer 1

If you also need to import files from parent directories you can add to the path as such:

import os
import sys
module_path = "/home/ec2-user/SageMaker/{module_name}"
if module_path not in sys.path:
    sys.path.append(module_path)

Then you can import as if you were in a normal python environment from the root of your project

Answer 2

You can add a __init__.py file to your package directory to make it a Python package. Then you will be import the modules from the package inside your Jupyter notebook

/home/ec2-user/SageMaker
    -- Notebook.ipynb 
    -- mypackage
        -- __init__.py
        -- mymodule.py

Contents of Notebook.ipynb

from mypackage.mymodule import SomeClass, SomeOtherClass

For more details, see https://docs.python.org/3/tutorial/modules.html#packages

Thanks for using Amazon SageMaker!

Answer 3

If you use SageMaker Studio, you need to take care about the path.

In the notebook of SageMaker Studio :

!df -h , You will see the line:

127.0.0.1:/200005  8.0E  1.3G  8.0E   1% /root

And !pwd will be:

/root

It is different from the bash terminal :

In bash terminal:

df -h
127.0.0.1:/200005  8.0E  1.4G  8.0E   1% /home/sagemaker-user

pwd
/home/sagemaker-user

So, the outside file path "/home/sagemaker-user" will be mapped into "/root" in your notebooks under SageMaker Studio.

So, the module path will change:

import sys
sys.path.append('/root/module_name')

The detail information can ref thislink