what is the difference between using a hugging face estimator with training script and directly using a notebook in AWS sagemaker?

Question

in tutorials like Fine-tuning a pytorch bert model and deploying it with sagemaker and fine-tune and host huggingface models on sagemaker , a hugging face estimator is used to call a training script. What would be the difference if I just directly ran the script's code in the notebook itself? is it because the estimator makes it easier to deploy the model?

Answer 1

You could run the script in the notebook itself but it would not deploy with SageMaker provided capabilities then. The estimator that you are seeing is what specifies to SageMaker what framework you are using and the training script that you are passing in. If you ran the script code in the notebook that would be like training in your local environment. By passing in the script to the Estimator you are running a SageMaker training job. The estimator is meant to encapsulate training on SageMaker.

SageMaker Estimator Overview: https://sagemaker.readthedocs.io/en/stable/overview.html