在 lambda 函数中调用 Sagemaker 端点 invoke_endpoint，请求正文格式的问题

Question

I have a deployed Sagemaker endpoint. When testing the endpoint using Predictor.predict , the endpoint works fine. I can pass down whichever Json format, it is able to process it correctly. However, I've been struggling calling endpoint from Lambda by using client.invoke_endpoint

I am trying to modify my request body to follow this format in this AWS documentation .

let request = {
  // Instances might contain multiple rows that predictions are sought for.
  "instances": [
    {
      // Request and algorithm specific inference parameters.
      "configuration": {},
      // Data in the specific format required by the algorithm.
      "data": {
         "<field name>": dataElement
       }
    }
  ]
}

I am not sure what should the configuration be, so this is what my request body looks like.

{
  "instances": [
    {
      "data": {
        "ID": "some ID",
        "ACCOUNT": null,
        "LEAD": some ID
        "FORMNAME": "some Form",
        "UTMMEDIUM": "some Medium",
        "UTMSOURCE": "some Source"
      }
    },
    {
      "data": {
        "ID": "some ID"
        "ACCOUNT": "some ID"
        "LEAD": null,
        "FORMNAME": "some Form"
        "UTMMEDIUM": null,
        "UTMSOURCE": null
      }
    }
  ]
}

This is my Lambda function

import os
import io
import boto3
import json

# grab environment variables
ENDPOINT_NAME = 'xxxxx'

client = boto3.client('sagemaker-runtime')
def lambda_handler(event, context):
    print("Received event: " + json.dumps(event, indent=2))
    data = json.loads(json.dumps(event))
    payload = str(data["instances"])
    response = client.invoke_endpoint(EndpointName = ENDPOINT_NAME,
                                      Body = payload,
                                      ContentType = 'application/json',
                                      Accept = 'application/json')
    print(response)
    return 'nothing'

I am able to invoke the Endpoint using the code above, but the endpoint kept having trouble processing the input. The input_fn in the endpoint looks like this

def input_fn(input_data, content_type):

    if content_type == 'text/csv':
        # Read the raw input data as CSV. 
        df = pd.read_csv(StringIO(input_data))

        return df
    
    elif content_type == 'application/json':
        print('input_fn (elif): input_data')
        print(input_data)
        print(type(input_data))
        print('input_fn (elif): input_data eval')
        print(eval(input_data))
        print('input_fn (elif): input_data eval type')
        print(type(eval(input_data)))
        df = pd.read_json(eval(input_data))
        print('input_fn (elif): df.columns')
        print(df.columns)
        return df
    
    else:
        raise ValueError("{} not supported by script!".format(content_type))

The error I got is ValueError: Invalid file path or buffer object type: <class 'list'>

The type of input_data is a string, and the type of eval(input_data) is a list.

I appreciate any insight! I've tried so many different things, including removing eval from my input_fn , or change pd.json_read to json.loads(json.dumps()) with pd.DataFrame.from_dict . I've gotten different errors like json.decoder.JSONDecodeError: Expecting value: Line Column 42 (column 42 is where the location of null), and unhashable type: 'dict'

I am really confused and not sure what to to next. Thank you!

Answer 1

This would involve debugging a bit more But I recommend replacing instances of input_data with input_data[0] and also try decoding since it could be byte encoded. Maybe something like this, the reason for this kind of recommendation is described at bottom and might help, also instead of print use logging so that you can see input_data value in cloudwatch logs

input_data = (input_data[0].get('data') or input_data[0].get('body')).decode('utf-8')

Reason for recommendation:

In Sagemaker when using pytorch type models

• the input_data is a list and each entry corresponds to data received from some request

• The size of list will be number of request in it which also correspond to batch size in case it is using torchserve. By default batch_size is 1 in torchserve, so input_data is single element list.

• also there is chance that you get byte encoded data

• You might refer here also https://pytorch.org/serve/custom_service.html

This should help explain ValueError: Invalid file path or buffer object type: <class 'list'> and JSON decode error

Answer 2

The issue you are having is in this line

payload = str(data["instances"])

You've already serialized the data, so the following code block will do justice on the boto3 invocation side as you properly decode the response.

#event is the test request you are sending
data = json.loads(json.dumps(event))
payload = json.dumps(data)
response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME,
                                   ContentType='application/json',
                                   Body=payload)
result = json.loads(response['Body'].read().decode()) #decode response

For processing the data you want appropriate exception handling when getting a non-JSON input. Use the following code to parse the request on the Flask end.

input_json = flask.request.get_json()
input = input_json['input'] #whatever your input json key is
result = model.predict(input)
# Transform predictions to JSON
result = {
        'output': predictions
}
resultjson = json.dumps(result)
return flask.Response(response=resultjson, status=200, mimetype='application/json')

I am contributing this on behalf of my employer, AWS. My contribution is licensed under the MIT license. See here for a more detailed explanation.

https://aws-preview.aka.amazon.com/tools/stackoverflow-samples-license/