![](/img/trans.png)
[英]xgboost in R: how does xgb.cv pass the optimal parameters into xgb.train
[英]how to specify train and test indices for xgb.cv in R package XGBoost
我最近發現了xgb.cv
的folds
參數,它允許指定驗證集的索引。 然后在xgb.cv
調用輔助函數xgb.cv.mknfold
,然后將每個折疊的剩余索引作為相應折疊的訓練集的索引。
問題:我可以通過 xgboost 接口中的任何接口指定訓練和驗證索引嗎?
我的主要動機是執行時間序列交叉驗證,我不希望“非驗證”索引被自動分配為訓練數據。 一個例子來說明我想做的事情:
# assume i have 100 strips of time-series data, where each strip is X_i
# validate only on 10 points after training
fold1: train on X_1-X_10, validate on X_11-X_20
fold2: train on X_1-X_20, validate on X_21-X_30
fold3: train on X_1-X_30, validate on X_31-X_40
...
目前,使用folds
參數會迫使我使用剩余的示例作為驗證集,這大大增加了誤差估計的方差,因為剩余數據的數量大大超過了訓練數據,並且可能與訓練數據具有非常不同的分布,尤其是對於較早的褶皺。 這就是我的意思:
fold1: train on X_1-X_10, validate on X_11-X100 # huge error
...
如果其他軟件包方便(即不需要我撬開源代碼)並且不會使原始 xgboost 實現中的效率無效,我願意接受其他軟件包的解決方案。
我認為問題的底部是錯誤的,應該說:
強迫我用剩下的例子作為訓練集
似乎提到的輔助函數xgb.cv.mknfold
也不再存在了。 注意我的xgboost版本是0.71.2
。
然而,這似乎可以通過對xgb.cv
一個小的修改來xgb.cv
,例如:
xgb.cv_new <- function(params = list(), data, nrounds, nfold, label = NULL,
missing = NA, prediction = FALSE, showsd = TRUE, metrics = list(),
obj = NULL, feval = NULL, stratified = TRUE, folds = NULL, folds_train = NULL,
verbose = TRUE, print_every_n = 1L, early_stopping_rounds = NULL,
maximize = NULL, callbacks = list(), ...) {
check.deprecation(...)
params <- check.booster.params(params, ...)
for (m in metrics) params <- c(params, list(eval_metric = m))
check.custom.obj()
check.custom.eval()
if ((inherits(data, "xgb.DMatrix") && is.null(getinfo(data,
"label"))) || (!inherits(data, "xgb.DMatrix") && is.null(label)))
stop("Labels must be provided for CV either through xgb.DMatrix, or through 'label=' when 'data' is matrix")
if (!is.null(folds)) {
if (!is.list(folds) || length(folds) < 2)
stop("'folds' must be a list with 2 or more elements that are vectors of indices for each CV-fold")
nfold <- length(folds)
}
else {
if (nfold <= 1)
stop("'nfold' must be > 1")
folds <- generate.cv.folds(nfold, nrow(data), stratified,
label, params)
}
params <- c(params, list(silent = 1))
print_every_n <- max(as.integer(print_every_n), 1L)
if (!has.callbacks(callbacks, "cb.print.evaluation") && verbose) {
callbacks <- add.cb(callbacks, cb.print.evaluation(print_every_n,
showsd = showsd))
}
evaluation_log <- list()
if (!has.callbacks(callbacks, "cb.evaluation.log")) {
callbacks <- add.cb(callbacks, cb.evaluation.log())
}
stop_condition <- FALSE
if (!is.null(early_stopping_rounds) && !has.callbacks(callbacks,
"cb.early.stop")) {
callbacks <- add.cb(callbacks, cb.early.stop(early_stopping_rounds,
maximize = maximize, verbose = verbose))
}
if (prediction && !has.callbacks(callbacks, "cb.cv.predict")) {
callbacks <- add.cb(callbacks, cb.cv.predict(save_models = FALSE))
}
cb <- categorize.callbacks(callbacks)
dall <- xgb.get.DMatrix(data, label, missing)
bst_folds <- lapply(seq_along(folds), function(k) {
dtest <- slice(dall, folds[[k]])
if (is.null(folds_train))
dtrain <- slice(dall, unlist(folds[-k]))
else
dtrain <- slice(dall, folds_train[[k]])
handle <- xgb.Booster.handle(params, list(dtrain, dtest))
list(dtrain = dtrain, bst = handle, watchlist = list(train = dtrain,
test = dtest), index = folds[[k]])
})
rm(dall)
basket <- list()
num_class <- max(as.numeric(NVL(params[["num_class"]], 1)),
1)
num_parallel_tree <- max(as.numeric(NVL(params[["num_parallel_tree"]],
1)), 1)
begin_iteration <- 1
end_iteration <- nrounds
for (iteration in begin_iteration:end_iteration) {
for (f in cb$pre_iter) f()
msg <- lapply(bst_folds, function(fd) {
xgb.iter.update(fd$bst, fd$dtrain, iteration - 1,
obj)
xgb.iter.eval(fd$bst, fd$watchlist, iteration - 1,
feval)
})
msg <- simplify2array(msg)
bst_evaluation <- rowMeans(msg)
bst_evaluation_err <- sqrt(rowMeans(msg^2) - bst_evaluation^2)
for (f in cb$post_iter) f()
if (stop_condition)
break
}
for (f in cb$finalize) f(finalize = TRUE)
ret <- list(call = match.call(), params = params, callbacks = callbacks,
evaluation_log = evaluation_log, niter = end_iteration,
nfeatures = ncol(data), folds = folds)
ret <- c(ret, basket)
class(ret) <- "xgb.cv.synchronous"
invisible(ret)
}
我剛剛添加了一個可選參數folds_train = NULL
並稍后以這種方式在函數內部使用它(見上文):
if (is.null(folds_train))
dtrain <- slice(dall, unlist(folds[-k]))
else
dtrain <- slice(dall, folds_train[[k]])
然后您可以使用該功能的新版本,例如如下所示:
# save original version
orig <- xgboost::xgb.cv
# devtools::install_github("miraisolutions/godmode")
godmode:::assignAnywhere("xgb.cv", xgb.cv_new)
# now you can use (call) xgb.cv with the additional argument
# once you are done, or may want to switch back to the original version
# (if you restart R you will also be back to the original version):
godmode:::assignAnywhere("xgb.cv", orig)
所以現在您應該能夠使用額外的參數調用該函數,為訓練數據提供額外的索引。
請注意,我沒有時間對此進行測試。
根據xgboost::xgb.cv
文檔,您可以通過folds
參數(默認為NULL
!)傳遞自定義測試索引。 它需要作為一個列表傳遞,其中每個元素都是一個索引向量。
例如,如果你想做一個時間序列的分裂,你可以這樣做:
create_test_idx <- function(size) {
half_size <- round(size / 2)
step <- round(0.1 * half_size)
starts <- seq(from = half_size, to = size - step, by = step)
return(lapply(starts, function(x) return(c(as.integer(x), as.integer(size)))))
}
my_custom_idx <- create_test_idx(nrow(my_train_data))
然后(例如),
xgbcv <- xgboost::xgb.cv(
params = params,
data = mydata,
nrounds = 10000,
folds = my_custom_idx,
showsd = T,
verbose = 0,
early_stopping_rounds = 200,
maximize = F
)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.