How to deploy a model trained with a custom package on Google ML Engine?

Question

I'm able to train a scikit-learn model that includes custom functions on Google ML Engine with a call like this:

gcloud ml-engine jobs submit training "module_name" \
  --job-dir gs://my_bucket/scikit_learn_job_dir \
  --package-path ./my_package/ \
  --module-name my_package.trainer.train \
  --region us-east1 \
  --runtime-version='1.12' \
  --python-version='3.5' \
  --scale-tier BASIC

This code trains the model and uploads a pickle file, model.pkl to gs://my_bucket/models .

However, I get an error when trying to deploy with :

gcloud ml-engine versions create "version_name" \
  --model model_name \
  --origin gs://my_bucket/models \
  --runtime-version='1.12' \
  --framework "SCIKIT_LEARN" \
  --python-version='3.5'

The error says that my_package cannot be loaded. Indeed, I can't find any option equivalent to --package-path in the version create command documentation .

Answer 1

This may be too late to respond, but if you trained the model with some custom package, you will also need to provide that same custom package during deployment (the pickle file alone does not have your custom code). That is probably why your deployment failed. Please see this for more information.