`scikit-optimize` package 中的 TypeError

Question

When I use scikit-optimize version 0.7.4 to optimize a scikit-learn 0.23 model:

    rf = BayesSearchCV(
        RandomForestClassifier(
            min_samples_leaf=0.01, oob_score=True
        ), {
            'n_estimators': Integer(30, 200),
            'max_depth': Integer(10, 150),
            'min_samples_split': Real(0.05, 0.3),
        }, n_iter=32
    )

When I run rf.fit , it says,

  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\space\space.py", line 764, in rvs
    if sp_version < (0, 16):
TypeError: '<' not supported between instances of 'Version' and 'tuple'

But when I simply use RandomForestClassifier , and fit it, the error doesn't occur. So, how to avoid this problem? Thank you!

The full of traceback is as following.

Traceback (most recent call last):
  File "C:/Users/cloudy/PyCharmProjects/clixove/BasicML/classifier.py", line 106, in <module>
    rf.fit(clf.data['X_train'], clf.data['Y_train'])
  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\searchcv.py", line 678, in fit
    optim_result = self._step(
  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\searchcv.py", line 552, in _step
    params = optimizer.ask(n_points=n_points)
  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\optimizer\optimizer.py", line 360, in ask
    x = opt.ask()
  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\optimizer\optimizer.py", line 332, in ask
    return self._ask()
  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\optimizer\optimizer.py", line 398, in _ask
    return self.space.rvs(random_state=self.rng)[0]
  File "C:\Users\cloudy\PyCharmProjects\clixove\venv\lib\site-packages\skopt\space\space.py", line 764, in rvs
    if sp_version < (0, 16):
TypeError: '<' not supported between instances of 'Version' and 'tuple'

Answer 1

If scikit-learn version is not important in your problem, you can downgrade scikit-learn version to '0.20.3' by pip install -U scikit-learn == 0.20.3

Answer 2

I've solved changing skopt/space/space.py lines 763-768

 for dim in self.dimensions:
            
            if sp_version < (0, 16):
                columns.append(dim.rvs(n_samples=n_samples))
            else:
                columns.append(dim.rvs(n_samples=n_samples, random_state=rng))

into

 for dim in self.dimensions:
            
            try:
                columns.append(dim.rvs(n_samples=n_samples, random_state=rng))
            except:
                columns.append(dim.rvs(n_samples=n_samples))

Answer 3

The problems with scikit-learn >= 0.23 have been fixed in version 0.8.1

Following is the PIP installation:

pip install scikit-optimize==0.8.1

Reference: scikit-optimize 0.8.1

Answer 4

I encountered the same issue. It looks like a new change to sci-kit learn changed how versions are read. Checkout the change here .

If setuptools is not installed, LooseVersion is used, which returns a Version type rather than a tuple.

Does installing setuptools in your environment with pip install setuptools solve this for you?