Amazon Sagemaker: User Input data validation in Inference Endpoint

Question

I have successfully built a Sagemaker endpoint using a Tensorflow model. The pre and post processing is done inside "inference.py" which calls a handler function based on this tutorial: https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/using_tf.html#how-to-implement-the-pre-and-or-post-processing-handler-s

My questions are:

• Which method is good for validating user input data within inference.py?
• If such validation tests fail (eg wrong data types or data not in allowed range, etc.), how is it possible to return appropriate error messages with status codes to the user?
• How is this compatible with the API gateway placed above the endpoint?

Here is the structure of the inference.py with the desired validation check as a comment:

import json
import requests


def handler(data, context):
    """Handle request.
    Args:
        data (obj): the request data
        context (Context): an object containing request and configuration details
    Returns:
        (bytes, string): data to return to client, (optional) response content type
    """
    processed_input = _process_input(data, context)
    response = requests.post(context.rest_uri, data=processed_input)
    return _process_output(response, context)


def _process_input(data, context):
    if context.request_content_type == 'application/json':
        # pass through json (assumes it's correctly formed)
        d = data.read().decode('utf-8')
        data_dict = json.loads(data)


        # ----->   if data_dict['input_1'] > 25000:
        # ----->       return some error specific message with status code 123


        return some_preprocessing_function(data_dict)

    raise ValueError('{{"error": "unsupported content type {}"}}'.format(
        context.request_content_type or "unknown"))


def _process_output(data, context):
    if data.status_code != 200:
        raise ValueError(data.content.decode('utf-8'))

    response_content_type = context.accept_header
    prediction = data.content
    return prediction, response_content_type

Answer 1

I will answer your questions inline below:

1. Which method is good for validating user input data within inference.py?

Seeing that you have a handler function, input_handler and output_handler are ignored. Thus, inside your handler function (as you are correctly doing) you can have the validation logic.

2. If such validation tests fail (eg wrong data types or data not in allowed range, etc.), how is it possible to return appropriate error messages with status codes to the user?

I like to think of my SageMaker endpoint as a web server. Thus, you can return any valid HTTP response code with a response message. Please see this example inference.py file that I found as a reference.

_return_error(
            415, 'Unsupported content type "{}"'.format(context.request_content_type or "Unknown")
        )

def _return_error(code, message):
    raise ValueError("Error: {}, {}".format(str(code), message))

3. How is this compatible with the API gateway placed above the endpoint?

Please see this link for details on Creating a machine learning-powered REST API with Amazon API Gateway mapping templates and Amazon SageMaker.