How to update sklearn in microsoft azure notebooks?

Question

I am new at Machine Learning and I am looking at some of the Scikit-Learn documentation. I use an account at Microsoft Azure https://notebooks.azure.com/ .

Now I am trying to run the code of this example:

https://scikit-learn.org/stable/auto_examples/cluster/plot_agglomerative_dendrogram.html .

I obtain the following error

---------------------------------------------------------------------------
   TypeError                                 Traceback (most recent call last)
    <ipython-input-23-fc0d954d99ec> in <module>
 44 
 45 # setting distance_threshold=0 ensures we compute the full tree.
---> 46 model = AgglomerativeClustering(distance_threshold=0, n_clusters=None)
 47 
 48 model = model.fit(X)

   TypeError: __init__() got an unexpected keyword argument 'distance_threshold'

I have done a search and the problem is that I have sklearn '0.20.3' version and in order to run that code I need the 0.22 version. I have been trying to update sklearn with

!pip install --update sklearn

and obtain this result:

Usage:   
pip install [options] <requirement specifier> [package-index-options] ...
pip install [options] -r <requirements file> [package-index-options] ...
pip install [options] [-e] <vcs project url> ...
pip install [options] [-e] <local project path> ...
pip install [options] <archive url/path> ...

no such option: --update

Searching the web it looks that there are some problem when trying to update sklearn in jupyter notebooks, but usually the answers are related to Anaconda Jupyter notebooks, not to Azure notebooks.

I will appreciate any hint to fix this problem.

Answer 1

To update sklearn try using the below command:

pip install -U scikit-learn 

This works for me.

Answer 2

https://notebooks.azure.com/help/jupyter-notebooks/package-installation

The

!pip !pip install scikit-learn==0.22.1

option did not work, even if it showed

Successfully installed joblib-0.14.1 scikit-learn-0.22.1

Instead,

!conda install scikit-learn==0.22.1  -y

was succesful, and I executed the code correctly. It took some minutes.

As stated, your packages will only be available for the lifetime of your notebook server and only in the current project.

Answer 3

这对我有用：

pip install --upgrade pandas

Answer 4

Stuff to pay attention to:

• Kernel needs to be restarted if you already imported sklearn (even with the %autoreload 2 magic).
• If you changed your kernel from the default (eg from Python 3.6 - AzureML to 3.8 ), your PATH may not have followed (see !which pip ) and you may be installing in the wrong kernel. To install in 3.8 I had to run:

! /anaconda/envs/azureml_py38/bin/pip install --user -U scikit-learn

Same trick for conda /anaconda/envs/azureml_py38/bin/conda install -y scikit-learn==0.24.2 . I don't believe conda will have any advantage in this case, except as an excuse for taking a coffee during its crawling dependency resolution.

Upgrading from 0.22 to 0.24.2 in Python 3.8 - Azure ML still yields some error messages, but you can safely ignore them if you're not using these packages (and maybe even if you are)

ERROR: raiwidgets 0.4.0 has requirement ipython==7.16.1, but you'll have ipython 7.18.1 which is incompatible.
ERROR: raiwidgets 0.4.0 has requirement jinja2==2.11.1, but you'll have jinja2 2.11.2 which is incompatible.