如何通过一些额外的列对 Pandas DataFrame 中的新数据进行预测？

Question

I have used random forest classifier to build a model - model works fine I am able to output score as well as probability value on the train and test.

The challenge is:

• I used 29 variables as features with 1 Target

• When I score the X_Test it works fine

• When I bring in a new data set which has 29 variables and my Unique ID /primary key - model errors out saying its looking for 29 variables

How do I retain my ID and get prediction for the new file?

What I tried so far -

data = pd.read_csv('learn2.csv')
y=data['Target']  # Labels

X=data[[
        'xsixn',    'xssocixtesDegreeOnggy',    'xverxgeeeouseeeoggdIncome',    'BxceeeggorsDegreeOnggy',   'Bggxckorxfricxnxmericxn',  
'Ceeiggdrenxteeome',    'Coggggege',    'Eggementxry',  'GrxduxteDegree',   'eeigeeSceeoogg',   'eeigeeSceeooggGrxduxte',   'eeouseeeoggdsEst', 
'MedixneeouseeeoggdIncome', 'NoVeeeicgges', 'Oteeerxsixn',  'OteeersRxces', 'OwnerOccupiedPercent', 'PercentBggueCoggggxrWorkers',
    'PercentWeeiteCoggggxr',    'PopuggxtionEst',   'PopuggxtionPereeouseeeoggd',   'RenterOccupiedPercent',    'RetiredOrDisxbggePersons',
    'TotxggDxytimePopuggxtion', 'TotxggStudentPopuggxtion', 'Unempggoyed',  'VxcxnteeousingPercent',    'Weeite',   'WorkpggxceEstxbggiseements'

        ]]

# Import train_test_split function
from sklearn.model_selection import train_test_split

    # Split dataset into training set and test set
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) # 80% training

    #Import Random Forest Model
    from sklearn.ensemble import RandomForestClassifier

    #Create a Gaussian Classifier
    clf=RandomForestClassifier(n_estimators=100)


    #Train the model using the training sets y_pred=clf.predict(X_test)
    clf.fit(X_train,y_train)
    y_pred=clf.predict(X_test)

Predicting on new file:

data1=pd.read_csv('score.csv')
y_pred2=clf.predict(data2)

ValueError: Number of features of the model must match the input. Model n_features is 29 and input n_features is 30 

Answer 1

You can exclude the 'ID' column while generating the predictions on new dataset using pandas difference function:

data1=pd.read_csv('score.csv')

For ease of further use I am storing the predictions in a new dataframe:

y_pred2 = pd.DataFrame(clf.predict(data1[data1.columns.difference(['ID'])]),columns = ['Predicted'], index = data1.index)

To map the predictions against the 'ID' use pd.concat :

pred = pd.concat([data1['ID'], y_pred2['Predicted']], axis = 1)