python regression: predicting model with new data

Question

I am trying to use new data to predict new outcomes, however, I am dealing with the following error:

ValueError: feature_names mismatch: ['time', 'x', 'y'] ['f0', 'f1', 'f2'] expected x, time, y in input data training data did not have the following fields: f0, f1, f2

I can't understand why, since I have 3 predictors and I am using exactly 3 values in my array.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import xgboost as xgb
import datetime
import seaborn as sns
from numpy import asarray

data=[[1, 1,2 ,5],
        [2, 5,5,6],
        [3, 4,6,6]
        ,[5, 6,5,6],
        [7,9,9,7],
        [8, 7,9,4]
        ,[9, 2,3,8],
        [2, 5,1,9],
        [2,2,10,9]
        ,[3, 8,2,8],
        [6, 5,4,10],
        [6, 8,5 ,10]]
df = pd.DataFrame(data, columns=['time','x','y','target'])
xgb_reg=xgb.XGBRegressor( n_estimators= 30, max_depth=8, eta= 0.1, colsample_bytree= 0.4, subsample= 0.4) #(n_estimators=250, max_depth=15, eta=0.1, subsample=0.4, colsample_bytree=0.4)
y = (df.target)
X=df.drop(['target'], axis = 1)
print('========1=============')
model=xgb_reg.fit(X,y)
prediction=model.predict(X)
new_data=[[10,10,10]]
new_data_asarray=asarray(new_data)
pred=model.predict(new_data_asarray)
print(pred)

Answer 1

It's because your model is expecting a pandas dataframe as input.

Simply convert your X dataframe to a numpy array before training as shown below.

import numpy as np
import pandas as pd
import xgboost as xgb


data = [
    [1, 1, 2, 5],
    [2, 5, 5, 6],
    [3, 4, 6, 6],
    [5, 6, 5, 6],
    [7, 9, 9, 7],
    [8, 7, 9, 4],
    [9, 2, 3, 8],
    [2, 5, 1, 9],
    [2, 2, 10, 9],
    [3, 8, 2, 8],
    [6, 5, 4, 10],
    [6, 8, 5, 10],
]
df = pd.DataFrame(data, columns=["time", "x", "y", "target"])
xgb_reg = xgb.XGBRegressor(
    n_estimators=30, max_depth=8, eta=0.1, colsample_bytree=0.4, subsample=0.4
)  # (n_estimators=250, max_depth=15, eta=0.1, subsample=0.4, colsample_bytree=0.4)
y = df.target
X = df.drop(["target"], axis=1)

X = X.to_numpy()

print("========1=============")
model = xgb_reg.fit(X, y)
prediction = model.predict(X)
new_data = [[10, 10, 10]]
new_data_asarray = np.asarray(new_data)
pred = model.predict(new_data_asarray)
print(pred)

Answer 2

xgb expects same type of data for training and test. Since you used a pandas dataframe for training but gave a numpy array in prediction, it gives an error. (further, it tries to make a dataframe out of that array with default column names f* as the error suggests).

Therefore a fix is to convert the array used in prediction to a frame with the column names taken from the training X dataframe:

new_data = [[10,10,10]]
new_data_as_frame = pd.DataFrame(new_data, columns=X.columns)
pred = model.predict(new_data_as_frame)

Answer 3

When I provide input as a dataframe, with specified column names, it works.

model = xgb_reg.fit(X, y)
prediction = model.predict(X)
new_data = [[10, 10, 10]]
new_data = pd.DataFrame(new_data, columns=['time', 'x', 'y'])
pred = model.predict(new_data)
print(pred)  # [6.3624153]