How do I make an SK Learn Classifier accept a 2D array as input for predictions?

Question

So I've made a model with mixed data types and used the recommended example from the SK Learn Docs using the column transformer to build the classifer.

https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html#sphx-glr-auto-examples-compose-plot-column-transformer-mixed-types-py

Since the input comes from a csv, and is converted to a Pandas Dataframe, it looks like the X_test, X_train, y_test, y_train are all dataframes too. Passing y_test to the clf.predict() function works fine, and I receive the predictions.

However I want to host this model Google cloud ML Engine which accepts a 2D array of instances in the predictions request API. How do I make my classifier adjust to and accept an array of inputs rather than a dataframe? I realize this may be fairly trivial, but struggling to find a solution.

Answer 1

To make your classifier compatible with Google Cloud Machine Learning Engine (CMLE), you'll need to separate out the preprocessor and the LogisticRegression classifier from the pipeline. You will need to perform the preprocessing logic client side, and the standalone classifier will be hosted on CMLE.

After reading in the csv file and defining the number and categorical transformers, you'll need to modify the training code as follows:

...

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

preprocessor = ColumnTransformer(
    transformers=[
        ('num', numeric_transformer, numeric_features),
        ('cat', categorical_transformer, categorical_features)])
model = LogisticRegression(solver='lbfgs')

X_train_transformed = preprocessor.fit_transform(X_train)
model.fit(X_train_transformed, y_train)
print("model score: %.3f" % model.score(preprocessor.transform(X_test), y_test))

You can export the model (using either pickle or joblib) and deploy it on CMLE. When constructing your json request to CMLE for prediction, you'll first need to preprocess your dataframe into a 2D array using: preprocessor.transform(X_test) .