用于模型托管的 AWS Sagemaker 与 ECS

Question

I have pretrained model artifacts stored in S3 buckets. I want to create a service that loads this model and uses it for inference.

I am working in AWS ecosystem and confused between using ECS vs Sagemaker for model deployment? What are some pros/cons for choosing one over other?

Answer 1

SageMaker has a higher price mark but it is taking a lot of the heavy lifting of deploying a machine learning model, such as wiring the pieces (load balancer, gunicorn, CloudWatch, Auto-Scaling...) and it is easier to automate the processes such as A/B testing.

If you have a strong team of DevOps that have nothing more important to do, you can build a flow that will be cheaper than the SageMaker option. ECS and EKS are doing at the same time a lot of work to make it very easy for you to automate the machine learning model deployments. However, they will always be more general purpose and SageMaker with its focus on machine learning will be easier for these use cases.

The usual pattern of using the cloud is to use the managed services early on as you want to move fast and you don't really know where are your future problems. Once the system is growing and you start feeling some pains here and there, you can decide to spend the time and improve that part of the system. Therefore, if you don't know the pros/cons, start with using the simpler options.