使用 shell 脚本运行 python 脚本时如何解决此错误

Question

I am trying to run a ML algorithm - KNN Regressor function. I have successfully run this code on jupyter notebook and on vs code.

I then use a shell script to run the python script. However, an error code below was generated. I am not sure what's the issue.

I am running on virtual env with the dependencies installed. Also have requirements.txt generated with the shell script reading the requirements.txt file.

Anyone able to assist me on this?

def knn_regressor(preprocess_lr, x_train, x_test, y_train, y_test):
    #combine pre-processing with ML algorithm - KNNRegression
    pipeline = make_pipeline(preprocess_lr, KNeighborsRegressor())
    params = {
        'kneighborsregressor__n_neighbors': range(2, 21),
        'kneighborsregressor__weights': ['uniform', 'distance']
        }
    model_3 = GridSearchCV(pipeline, params, cv=5, scoring='neg_mean_squared_error')
    #train the pipeline
    model_3.fit(x_train, y_train)
    #fit the model on the test data
    pred_test = model_3.predict(x_test)
    #display the results of the metrics
    rmse_model = np.sqrt(mean_squared_error(y_test, pred_test))
    r2_model = r2_score(y_test, pred_test)
    print("..........")
    print("Results on Test Data for KNN Regressor")
    print("RMSE - KNN Regressor: {:.2f}".format(rmse_model))
    print("R2 Score - KNN Regressor: {:.5f}".format(r2_model))

    if(int(sys.argv[1]) == 1):
        print("Applying KNN Regressor algorithm for prediction...")
        knn_regressor(preprocess_lr, x_train, x_test, y_train, y_test)

shell script:

#!/usr/bin/env bash

pip install -r requirements.txt

echo "Please select your algorithm: "
echo "1 - KNN Regressor"

python src/module1.py $user_algo

Error message:

 knn_regressor(preprocess_lr, x_train, x_test, x_train, y_test)
  File "src/module1.py", line 62, in knn_regressor
    model_3.fit(x_train, y_train)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\utils\validation.py", line 72, in inner_f
    return f(**kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_search.py", line 736, in fit
    self._run_search(evaluate_candidates)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_search.py", line 1188, in _run_search
    evaluate_candidates(ParameterGrid(self.param_grid))
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_search.py", line 708, in evaluate_candidates
    out = parallel(delayed(_fit_and_score)(clone(base_estimator),
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 1029, in __call__
    if self.dispatch_one_batch(iterator):
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 847, in dispatch_one_batch
    self._dispatch(tasks)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 765, in _dispatch
    job = self._backend.apply_async(batch, callback=cb)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\_parallel_backends.py", line 208, in apply_async
    result = ImmediateResult(func)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\_parallel_backends.py", line 572, in __init__
    self.results = batch()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 252, in __call__
    return [func(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\joblib\parallel.py", line 252, in <listcomp>
    return [func(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_validation.py", line 560, in _fit_and_score
    test_scores = _score(estimator, X_test, y_test, scorer)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\model_selection\_validation.py", line 607, in _score
    scores = scorer(estimator, X_test, y_test)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\metrics\_scorer.py", line 87, in __call__
    score = scorer._score(cached_call, estimator,
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\metrics\_scorer.py", line 206, in _score
    y_pred = method_caller(estimator, "predict", X)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\metrics\_scorer.py", line 53, in _cached_call
    return getattr(estimator, method)(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\utils\metaestimators.py", line 119, in <lambda>
    out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\pipeline.py", line 408, in predict
    return self.steps[-1][-1].predict(Xt, **predict_params)
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\sklearn\neighbors\_regression.py", line 185, in predict
    y_pred = np.mean(_y[neigh_ind], axis=1)
  File "<__array_function__ internals>", line 5, in mean
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\numpy\core\fromnumeric.py", line 3372, in mean
    return _methods._mean(a, axis=axis, dtype=dtype,
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\numpy\core\_methods.py", line 162, in _mean
    ret = um.true_divide(
TypeError: unsupported operand type(s) for /: 'str' and 'int'

Answer 1

Looks like you might be using a different version of Python (or some of the libraries listed in requirements.txt ) to run the code in in your script, versus when you run with the other methods.

to fix, update the script to give the full path to the specific version of the python interpreter that you are using with the other environments. Also make sure you do the same with pip when running the install. And make sure the versions of all the libraries that are listed in requirements.txt are the same as those being used in the other environments.