Environment conflict within an azure ML pipeline

Question

I've been using my model training pipeline on azure until now with no issues. Last week I launched it and got an error regarding environment version conflicts. I changed nothing and yet it doesn't work anymore. How can I solve this?

from azureml.train.automl import automl\nImportError: cannot import name 'automl' from 'azureml.train.automl'

I also started getting this warning at the same time.

WARNING:azureml.pipeline.core.run:Expected a StepRun object but received <class 'azureml.core.run.Run'> instead.
This usually indicates a package conflict with one of the dependencies of azureml-core or azureml-pipeline-core.
Please check for package conflicts in your python environment

This is my environment notebook block:

from azureml.core.runconfig import RunConfiguration
from azureml.core.conda_dependencies import CondaDependencies

aml_run_config = RunConfiguration()
# Use just-specified compute target ("cpu-cluster")
aml_run_config.target = compute_target

# Specify CondaDependencies obj, add necessary packages
aml_run_config.environment.python.conda_dependencies = CondaDependencies.create(
    conda_packages=['pandas','scikit-learn','pyodbc'], 
    pip_packages=['azureml-sdk[automl]','pyarrow', 'azureml-core>=1.42.0', 'msrest==0.6.21', 'xgboost'])

I tried changing the versioning for the azureml-core, sdk, mrest etc but it still gives me the same error.

Answer 1

from azureml.train.automl import automl

The above syntax is used to import the automl. But due to updated versions, it is not the procedure to use Automl features. The below two procedures are the code blocks which can be useful to replace the existing work.

from azure.ai.ml import automl, Input

The above code block can be used to retrieve all the features required to perform autoML training. After using the above syntax

# training data from your local directory
my_training_data_input = Input(
    type=AssetTypes.MLTABLE, path="./data/training-mltable-folder"
)

# Remote MLTable definition
my_training_data_input  = Input(type=AssetTypes.MLTABLE, path="azureml://datastores/workspaceblobstore/paths/Classification/Train")
# note that the below is a code snippet -- you might have to modify the variable values to run it successfully
classification_job = automl.classification(
    compute=my_compute_name,
    experiment_name=my_exp_name,
    training_data=my_training_data_input,
    target_column_name="y",
    primary_metric="accuracy",
    n_cross_validations=5,
    enable_model_explainability=True,
    tags={"my_custom_tag": "My custom value"}
)

# Limits are all optional

classification_job.set_limits(
    timeout_minutes=600, 
    trial_timeout_minutes=20, 
    max_trials=5,
    enable_early_termination=True,
)

# Training properties are optional
classification_job.set_training(
    blocked_training_algorithms=["LogisticRegression"], 
    enable_onnx_compatible_models=True
)

The above are the properties to use in the training model.

The is another approach to achieve similar work.

Using AutoMLRun class can help to run the model.
from azureml.train.automl.run import AutoMLRun
   ws = Workspace.from_config()
   experiment = ws.experiments['my-experiment-name']
   automl_run = AutoMLRun(experiment, run_id = your run ID')