Automate the execution of a .ipynb file in AWS SageMaker by Websocket

Question

I have an issue. At November 10th before, my lambda code use websocket to communicate sagemaker which is okay to automatic execution of a.ipynb file. The code is below

import boto3
import time
from botocore.vendored import requests
import websocket

def lambda_handler(event, context):
    sm_client = boto3.client('sagemaker')
    notebook_instance_name = 'test'
    url = sm_client.create_presigned_notebook_instance_url(NotebookInstanceName=notebook_instance_name)['AuthorizedUrl']

    url_tokens = url.split('/')
    http_proto = url_tokens[0]
    http_hn = url_tokens[2].split('?')[0].split('#')[0]

    s = requests.Session()
    r = s.get(url)
    cookies = "; ".join(key + "=" + value for key, value in s.cookies.items())

    ws = websocket.create_connection
      ("wss://{}/terminals/websocket/1".format(http_hn),  
      cookie=cookies, 
      host=http_hn, 
      origin=http_proto + "//" + http_hn,
      header=["User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36"])


    ws.send("""[ "stdin", "jupyter nbconvert --execute --to notebook --inplace /home/ec2-user/SageMaker/Scikit.ipynb --ExecutePreprocessor.kernel_name=python3 --ExecutePreprocessor.timeout=1500\\r" ]""")
    time.sleep(1)
    ws.close()
    return None

But when November 18th, I discovered that websocket connects getting the error. The error code is WebSocketBadStatusException: Handshake status 200 OK. I can't figure out for a few days. Does anyone have a suggestion / solution?

I survey this article for many days. But I don't get the answer.

Answer 1

You could look at automating your notebook using SageMaker Processing Jobs. Kindly see this blog post here: https://aws.amazon.com/blogs/machine-learning/scheduling-jupyter-notebooks-on-sagemaker-ephemeral-instances/