带R的xgboost的问题

Question

I used xgboost to do logistic regression. I followed the steps from , but I got two problems.The datasets are found here .

First, when I run the follow code:

bst <- xgboost(data = sparse_matrix, label = output_vector,nrounds = 39,param)

Then, I got

 [0]train-rmse:0.350006
 [1]train-rmse:0.245008
 [2]train-rmse:0.171518
 [3]train-rmse:0.120065
 [4]train-rmse:0.084049
 [5]train-rmse:0.058835
 [6]train-rmse:0.041185
 [7]train-rmse:0.028830
 [8]train-rmse:0.020182
 [9]train-rmse:0.014128
[10]train-rmse:0.009890
[11]train-rmse:0.006923
[12]train-rmse:0.004846
[13]train-rmse:0.003392
[14]train-rmse:0.002375
[15]train-rmse:0.001662
[16]train-rmse:0.001164
[17]train-rmse:0.000815
[18]train-rmse:0.000570
[19]train-rmse:0.000399
[20]train-rmse:0.000279
[21]train-rmse:0.000196
[22]train-rmse:0.000137
[23]train-rmse:0.000096
[24]train-rmse:0.000067
[25]train-rmse:0.000047
[26]train-rmse:0.000033
[27]train-rmse:0.000023
[28]train-rmse:0.000016
[29]train-rmse:0.000011
[30]train-rmse:0.000008
[31]train-rmse:0.000006
[32]train-rmse:0.000004
[33]train-rmse:0.000003
[34]train-rmse:0.000002
[35]train-rmse:0.000001
[36]train-rmse:0.000001
[37]train-rmse:0.000001
[38]train-rmse:0.000000

train-rmse is finally equal to 0! Is that normal? Usually,I know train-rmse can't be equal to 0. So,where is my problem?

Second, when I run

importance <- xgb.importance(sparse_matrix@Dimnames[[2]], model = bst)

Then, I got a Error:

Error in eval(expr, envir, enclos) : object 'Yes' not found.

I don't know what does it mean, maybe the first question leads to the second one.

library(data.table)
train_x<-fread("train_x.csv")
str(train_x)
train_y<-fread("train_y.csv")
str(train_y)
train<-merge(train_y,train_x,by="uid")
train$uid<-NULL
test<-fread("test_x.csv")
require(xgboost)
require(Matrix)
sparse_matrix <- sparse.model.matrix(y~.-1, data = train)
head(sparse_matrix)
output_vector = train[,y] == "Marked"
param <- list(objective = "binary:logistic", booster = "gblinear",
          nthread = 2, alpha = 0.0001,max.depth = 4,eta=1,lambda = 1)
bst <- xgboost(data = sparse_matrix, label = output_vector,nrounds = 39,param)
importance <- xgb.importance(sparse_matrix@Dimnames[[2]], model = bst)

Answer 1

I ran into the same problem (Error in eval(expr, envir, enclos) : object 'Yes' not found.) and the reason was the following:

I tried to do

dt = data.table(x = runif(10), y = 1:10, z = 1:10)
label = as.logical(dt$z)
train = dt[, z := NULL]
trainAsMatrix = as.matrix(train)
label = as.matrix(label)

bst <- xgboost(data = trainAsMatrix, label = label, max.depth = 8,
               eta = 0.3, nthread = 2, nround = 50, objective = "reg:linear")
bst$featureNames = names(train)
xgb.importance(model = bst)

The problem comes from the line

label = as.logical(dt$z)

I got this line in there because the last time I used xgboost, I wanted to predict a categorial variable. Now since I want to do regression it should read:

label = dt$z

Maybe something similar causes the problem in your case?

Answer 2

Perhaps this is of any help. I'm often getting the same error when the labels have zero variation. Using the current CRAN version of xgboost, which is somewhat old already (0.4.4). xgb.train happily accepts this (showing a .50 AUC) but the error then shows when calling xgb.importance.

Cheers

Otto

[0] train-auc:0.500000  validate-auc:0.500000
[1] train-auc:0.500000  validate-auc:0.500000
[2] train-auc:0.500000  validate-auc:0.500000
[3] train-auc:0.500000  validate-auc:0.500000
[4] train-auc:0.500000  validate-auc:0.500000

[1] "XGB error: Error in eval(expr, envir, enclos): object 'Yes' not found\n"