如何在AWS SageMaker中加载训练集以构建模型？

Question

I am very new to SageMaker. Upon my first interaction, it looks like the AWS SageMaker requires you to start from its Notebook. I have a training set which is ready. Is there a way to bypass setting the Notebook and just to start by upload the training set? Or it should be done through the Notebook. If anyone knows some example fitting my need above, that will be great.

Answer 1

Amazon SageMaker is a combination of multiple services that each is independent of the others. You can use the notebook instances if you want to develop your models in the familiar Jupyter environment. But if just need to train a model, you can use the training jobs without opening a notebook instance.

There a few ways to launch a training job:

• Use the high-level SDK for Python that is similar to the way that you start a training step in your python code

kmeans.fit(kmeans.record_set(train_set[0]))

Here is the link to the python library: https://github.com/aws/sagemaker-python-sdk

• Use the low-level API to Create-Training-Job, and you can do that using various SDK (Java, Python, JavaScript, C#...) or the CLI.

sagemaker = boto3.client('sagemaker') sagemaker.create_training_job(**create_training_params)

Here is a link to the documentation on these options: https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-train-model-create-training-job.html

• Use Spark interface to launch it using a similar interface to creating an MLLib training job

val estimator = new KMeansSageMakerEstimator( sagemakerRole = IAMRole(roleArn), trainingInstanceType = "ml.p2.xlarge", trainingInstanceCount = 1, endpointInstanceType = "ml.c4.xlarge", endpointInitialInstanceCount = 1) .setK(10).setFeatureDim(784)

val model = estimator.fit(trainingData)

Here is a link to the spark-sagemaker library: https://github.com/aws/sagemaker-spark

• Create a training job in the Amazon SageMaker console using the wizard there: https://console.aws.amazon.com/sagemaker/home?region=us-east-1#/jobs

Please note that there a few options also to train models, either using the built-in algorithms such as K-Means, Linear Learner or XGBoost (see here for the complete list: https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html ). But you can also bring your own models for pre-baked Docker images such as TensorFlow ( https://docs.aws.amazon.com/sagemaker/latest/dg/tf.html ) or MXNet ( https://docs.aws.amazon.com/sagemaker/latest/dg/mxnet.html ), your own Docker image ( https://docs.aws.amazon.com/sagemaker/latest/dg/your-algorithms-training-algo.html ).