Sagemaker API to list Hyperparameters

Question

I'm currently trying to implement MLFlow Tracking into my training pipeline and would like to log the hyperparameters of my hyperparameter Tuning of each training job.

Does anyone know, how to pull the list of hyperparameters that can be seen on the sagemaker training job interface (on the AWS console)? Is there any other smarter way to list how models perform in comparison in Sagemaker (and displayed)?

I would assume there must be an easy and Pythonic way to do this (either boto3 or the sagemaker api) to get this data. I wasn't able to find it in Cloudwatch.

Many thanks in advance!

Answer 1

there is indeed a rather pythonic way in the SageMaker python SDK:

tuner = sagemaker.tuner.HyperparameterTuner.attach('< your tuning jobname>')

results = tuner.analytics().dataframe()  # all your tuning metadata, in pandas!

See full example here https://github.com/aws-samples/amazon-sagemaker-tuneranalytics-samples/blob/master/SageMaker-Tuning-Job-Analytics.ipynb

Answer 2

For doing more comparisons, go with what Oliver_Cruchant posted.

To just get the hyperparameters with the SageMaker Python SDK (v1.65.0+):

tuner = sagemaker.tuner.HyperparameterTuner.attach('your-tuning-job-name')
job_desc = tuner.describe()
job_desc['HyperParameterRanges']  # returns a dictionary with your tunable hyperparameters
job_desc['StaticHyperParameters']  # returns a dictionary with your other hyperparameters

and with boto3:

sagemaker = boto3.client('sagemaker')
job_desc = sagemaker.describe_hyper_parameter_tuning_job(HyperParameterTuningJobName='your-tuning-job-name')
job_desc['HyperParameterRanges']  # returns a dictionary with your tunable hyperparameters
job_desc['StaticHyperParameters']  # returns a dictionary with your other hyperparameters

Both ways return the result of calling the DescribeHyperParameterTuningJob API.

DescribeHyperParameterTuningJob API documentation: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeHyperParameterTuningJob.html