SageMaker Model Compiler for AWS App Runner

Question

Can SageMaker model compiler be used to optimize model for AWS App Runner? Based on documentation https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_OutputConfig.html#sagemaker-Type-OutputConfig-CompilerOptions it looks like it can support CPU devices - both X86 and ARM. Would the output model work with CPU-based App Runner?

Thanks very much for any insight if anyone tried it before.

Answer 1

You are probably referring to SageMaker Neo which is used for inference, rather to SageMaker Training Compiler (used for training).

SageMaker Neo can compile the model to a target CPU architecture, AppRunner is a high level service for light containers designed with simplicity as a primary goal. In App Runner you can set the num of vCPUs , I couldn't find any guarantees on which CPU model will be used to run the container (no surprise really). In practice, you could try optimizing for Intel instructions set, and see if the resulting runtime is supported on app runner.
Or if you really need more control and performance, switch to AWS Fargate and explicitly choose x86 or ARM.