在 AWS Lambda 上运行 Tensorflow 2 预测

Question

I want to run a Tensorflow 2 LinearClassifier on AWS Lambda. I have a trained LinearClassifier tensorflow model. I want to run only the prediction in AWS Lambda, because of cost efficeny.

Currently I'm not sure how to handle it.

• I tried to use normal TF2, but it is too big for AWS Lambda Layers. All descriptions you find are with older TF 1.1x versions and Python 2.x.
• Then I tried TensorflowJS, but it looks like the LinearClassifier using functions which are not implemented in TFJS.
• Last I tried to user Tensorflow Lite, but there I currently have problems to convert my model to TFLite.

Do you have an idea, how I can get TF 2 Estimation of LinearClassifier running on AWS Lambda?

Normally I think not all of the 1.5 GB Tensorflow modules are used for the simple prediction of an existing model. So is there a possibility to only get the files which are used for running the script?

Answer 1

I have found possible answers for your question:

1. To transform in TFlite model:

• a blog about "How to convert a Tensorflow Estimator to Tensorflow Lite": https://medium.com/datadriveninvestor/how-to-convert-a-tensorflow-estimator-to-tensorflow-lite-a3509a9ba902
• an other example with Keras model but maybe the same trick with get_concrete_function (it works for me with my keras model): https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/lite/examples/experimental_new_converter/Keras_LSTM_fusion_Codelab.ipynb

2. To use with AWS Lambda, there are a lot of blogs but none of them is perfect. For my case, I used tensorflow 2.3.0 to create my keras model. So I have to modified a little this very good tutorial:

• https://www.edeltech.ch/tensorflow/machine-learning/serverless/2020/07/11/how-to-deploy-a-tensorflow-lite-model-on-aws-lambda.html or his github: https://github.com/edeltech/tensorflow-lite-on-aws-lambda

For the part: "Clone TensorFlow from the official GitHub repository", we need to force a version of tensorflow because sometimes latest version is not compiling well.

For my case, I want tensorflow v2.3.0, so i modified this line:

git clone https://github.com/tensorflow/tensorflow.git

by

git clone --branch v.2.3.0 https://github.com/tensorflow/tensorflow.git

I have modified also at the end, to invoke lambda on AWS:

sls invoke local -f predict

by

sls invoke -f predict

I recommend you to do all the tutorial with the sample model provided to be sure everything is working.