How to change Sklearn flavors version in mlflow on azure machine learning?

Question

I need to change the flavors "sklearn_version" in mlflow from "0.22.1" to "1.0.0" on azure machine learning when I log my trained model, since this model will be incompatible with the sklearn version that I am using for deployment during inference. I could change the version of sklearn in conda.yml file by setting "conda_env" in

mlflow.sklearn.log_model(conda_env= 'my_env')

here is the screen shot of requirements.txt

however, sklearn version under flavors in MLmodel file remains unchanged and that is the file that causes problem:

and here is script that I use to create this mlflow experiment in azure machine learning notebooks.

import mlflow
from sklearn.tree import DecisionTreeRegressor

from azureml.core import Workspace
from azureml.core.model import Model
from azureml.mlflow import register_model


def run_model(ws, experiment_name, run_name, x_train, y_train):
    
    # set up MLflow to track the metrics
    mlflow.set_tracking_uri(ws.get_mlflow_tracking_uri())
    mlflow.set_experiment(experiment_name)  
    
    with mlflow.start_run(run_name=run_name) as run:
        
        # fit model
        regression_model = DecisionTreeRegressor()
        regression_model.fit(x_train, y_train)
    
        # log training score 
        training_score = regression_model.score(x_train, y_train)
        mlflow.log_metric("Training score", training_score)

        my_conda_env = {
                    "name": "mlflow-env",
                    "channels": ["conda-forge"],
                    "dependencies": [
                        "python=3.8.5",
                        {
                            "pip": [
                                "pip",
                                "scikit-learn~=1.0.0",
                                "uuid==1.30",
                                "lz4==4.0.0",
                                "psutil==5.9.0",
                                "cloudpickle==1.6.0",
                                "mlflow",
                            ],
                        },
                    ],
                }

        
        # register the model
        mlflow.sklearn.log_model(regression_model, "model", conda_env=my_conda_env)

    model_uri = f"runs:/{run.info.run_id}/model"
    model = mlflow.register_model(model_uri, "sklearn_regression_model")

if __name__ == '__main__':

    # connect to your workspace
    ws = Workspace.from_config()

    # create experiment and start logging to a new run in the experiment
    experiment_name = "exp_name"

    # mlflow run name
    run_name= '1234'

  
    # get train data
    x_train, y_train  = get_train_data()
    
    run_model(ws, experiment_name, run_name, x_train, y_train)

Any idea how can change the flavor sklearn version in MLmodel file from "0.22.1" to "1.0.0" in my script?

With many thanks in advance!

Answer 1

The modifications of the versions must be from "requirements.txt" . We need to manually override the versions we need and move the build to the pipeline.

Manually edit the following code

conda_env = {
    'channels': ['conda-forge'],
    'dependencies': [
        'python=3.8.8',
        'pip'],
    'pip': [
        'mlflow',
        'scikit-learn==0.23.2',
        'cloudpickle==1.6.0'
    ],
    'name': 'mlflow-env'
}
mlflow.sklearn.log_model(model, "my_model", conda_env=conda_env)

The below code should be in conda.yml

name: mlflow-env
channels:
  - conda-forge
dependencies:
- python=3.8.8
- pip
- pip:
  - mlflow
  - scikit-learn==0.23.2
  - cloudpickle==1.6.0

The following code block must be there in python_env.yaml

python: 3.8.8
build_dependencies:
  - pip==21.1.3
  - setuptools==57.4.0
  - wheel==0.37.0
dependencies:
  - -r requirements.txt

The following thing must be there in requirements.txt

mlflow
scikit-learn==0.23.2
cloudpickle==1.6.0

Answer 2

I was finally able to solve this issue. Apparently flavors within mlflow MLfile use the version of installed scikit-learn within the workspace. all I needed to to was to upgrade the scikit-learn from cli within workspace.

1. open cli in azure machine learning notebooks
2. upgarde your intended flavor package. My case was scikit-learn, in my case I used 1.0.2:

 pip install --upgrade scikit-learn==1.0.2

3. log your model in mlflow by running your training script one more time.