How to upload files in with a Sagemaker notebook instance created through AWS cloud formation template

Question

I am working on creating an AWS cloud formation stack wherein we create resources through a template. yaml and also create the folder for that resource in the project file to indicate what all files will go in that resource once it is created.

For example, I create a lambda function in the template. yaml with the name - "count_calories" and create a folder in the project file saying- "count_calories" and put a py file with lambda handler in it and requirement. txt file in it.

In a similar way, now I have to create a sagemaker notebook instance through the template.yaml and then upload jupyter notebooks in that notebook instance, each time the stack is created with that cloud formation template.

I have created the sagemaker notebook instance with the following template code:

 NotebookInstance: #Sagemaker notebook instance 
    Type: AWS::SageMaker::NotebookInstance
    Properties: 
      InstanceType: ml.t3.medium
      NotebookInstanceName: !Sub Calorie-NotebookInstance-${EnvVar}
      RoleArn: <RoleARN>
      RootAccess: Enabled
      VolumeSizeInGB: 200

I have 4 Jupyter notebooks and a data file that should go into this notebook instance once it is created. I want to do the upload through the code, not from the AWS console. Please suggest to me the right way to do it or point me to any example I can follow.

Many Thanks

Answer 1

You're on the right path by using template Type: AWS::SageMaker::NotebookInstance

Follow along the example here to create a SageMaker notebook using CFT

Answer 2

Consider using AWS::SageMaker::NotebookInstanceLifecycleConfig : https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sagemaker-notebookinstancelifecycleconfig.html

1. First, you need to refer the LifecycleConfigName in AWS::SageMaker::NotebookInstance resource, by name. That's why I'm using !GetAtt function and not !Ref .
2. Then, you need to create a resource AWS::SageMaker::NotebookInstanceLifecycleConfig that you referred in previous step.
3. Finally, in the line Fn::Base64: you insert the commands for code/file download. I'm using wget in this example, but you can probably use another bash commands, or even download a more complex script and run it. Consider the script should run in no more than 5 minutes: https://docs.aws.amazon.com/sagemaker/latest/dg/notebook-lifecycle-config.html

Please see next code example:

      JupyterNotebookInstance:
        Type: AWS::SageMaker::NotebookInstance
        Properties:
          InstanceType: ml.t3.medium
          RoleArn: !GetAtt JupyterNotebookIAMRole.Arn
          NotebookInstanceName: !Ref NotebookInstanceName
          LifecycleConfigName: !GetAtt JupyterNotebookInstanceLifecycleConfig.NotebookInstanceLifecycleConfigName
      JupyterNotebookInstanceLifecycleConfig:
        Type: "AWS::SageMaker::NotebookInstanceLifecycleConfig"
        Properties:
          OnStart:
            - Content:
                Fn::Base64: "cd /home/ec2-user/SageMaker/ && wget <your_files_url_here>"