How do I swap accuracy with another performance metric from the score() function after fitting the ML model in Pipeline with scikit-learn?

Question

Here is a part of my code:

def ml_pipeline(self):
    if self.control_panel['ml_pipeline_switch']:
        self.model = make_pipeline(self.preprocessor, self.control_panel['ml_algo'][1])
        self.model.fit(self.X_train, self.y_train)

def ml_pipeline_result(self, show_control_panel_switch=True): 
    if self.control_panel['ml_pipeline_switch']:
        print('Model score (training set): %.3f' % self.model.score(self.X_train, self.y_train))
        print('Model score (test set): %.3f' % self.model.score(self.X_test, self.y_test))

The score() seems to be producing accuracy. How can I swap accuracy with another performance metrics such as F1-macro or recall-macro ? I couldn't find anything in the document .

Answer 1

The short answer to your question is no, unless you hack your way through and redefine/overwrite the scikit-learn functions.

When you are using pipe.score() , it calls the score method from the classifier that is in the end of the pipeline.

Now, what is happening under the hood is that all classifiers in scikit-learn are based on the ClassifierMixin class, for which .score() is defined through accuracy_score , and this is hard-coded (see here ).