how to log metrics using kubeflow on google ai platform notebooks

Question

I am building ml models using google cloud platform's ai platform notebooks.

I know if I use ai platform jobs, it logs hyperparameters, metrics, etc with nice visualization but is there a way to create the same or similar structure so that I can log metrics etc?

Idealy I want to use kubeflow and not other tools like mlflow.

Answer 1

You would like to explore MLMD https://www.tensorflow.org/tfx/tutorials/mlmd/mlmd_tutorial

Edit: Interesting article: https://containerjournal.com/kubeconcnc/the-most-crucial-component-in-an-ml-pipeline-is-invisible/