使用 Blazingtext 预处理 Sagemaker 推理管道的数据

Question

I'm trying to figure out the best way to preprocess my input data for my inference endpoint for AWS Sagemaker. I'm using the BlazingText algorithm.

I'm not really sure the best way forward and I would be thankful for any pointers.

I currently train my model using a Jupyter notebook in Sagemaker and that works wonderfully, but the problem is that I use NLTK to clean my data (Swedish stopwords and stemming etc):

import nltk
nltk.download('punkt')
nltk.download('stopwords')

So the question is really, how do I get the same pre-processing logic to the inference endpoint?

I have a couple of thoughts about how to proceed:

• Build a docker container with the python libs & data installed with the sole purpose of pre-processing the data. Then use this container in the inference pipeline.

• Supply the Python libs and Script to an existing container in the same way you can do for external lib an notebook

• Build a custom fastText container with the libs I need and run it outside of Sagemaker.

• Will probably work, but feels like a "hack": Build a Lambda function that has the proper Python libs&data installed and calls the Sagemaker Endpoint. I'm worried about cold start delays as the prediction traffic volume will be low.

I would like to go with the first option, but I'm struggling a bit to understand if there is a docker image that I could build from, and add my dependencies to, or if I need to build something from the ground up. For instance, would the image sagemaker-sparkml-serving:2.2 be a good candidate?

But maybe there is a better way all around?

Answer 1

Niclas

I would suggest you try out BentoML .

BentoML is an open-source framework for high-performance model serving. It turns ML model into a production API endpoint with just a few lines of code.

For your use case:

In your notebook on Sagemaker, after you trained your model, You can define a prediction service spec with BentoML. This code builds a BentoService that expect a PickleArtifact, expose an API endpoint called predict, and automatically include pip dependencies

%%write my_service.py

from bentoml import api, BentoService, artifacts, env
from bentoml.artifacts import PickleArtifact
from bentoml.handlers import DataFrameHandler

@env(auto_pip_dependencies=True)
@artifacts([PickleArtifact('my_model')]
class MyTextClassifier(BentoService):
    def preprocess(self, raw_data):
        ...
        return processed_data;

    @api(DataframeHandler)
    def predict(self, df):
        processed_data = self.preprocess(df)
        return self.artifacts.my_model.predict(processed_data)

In the next cell, load the class defined from the previous cell and save the model.

from my_service import MyTextClassification

service = MyTextClassification()
service.pack('my_model', trained_model)
service.save()
# could also saved the model to S3 bucket
# service.save('s3://my_bucket')

You can easily deploy to Sagemaker within the notebook.

!bentoml sagemaker deploy my-deployment -b {service.name}:{service.version} --api-name predict

Here is an example of that. In this example, it is using Keras to build a text classification model and then deploy the model to Sagemaker for model serving.

https://github.com/bentoml/gallery/blob/master/keras/text-classification/keras-text-classification.ipynb

Disclaimer: I am one of the authors for BentoML.