[英]Extraction of tuned hyperparameters from tuning instance archive
我基於以下示例構建了一個自動化機器學習系統:
我使用了學習器 xgboost 和 Random Forest 並使用了branching
。 xgboost 在訓練階段給了我最好的結果。 因此,我提取了優化的超參數並構建了最終的 xgboost model:
lrn = as_learner(graph)
lrn$param_set$values = instance$result_learner_param_vals
我也對性能最好的隨機森林 model 的param_vals
感興趣。
我想,我可以獲得超參數並像這樣保存最好的隨機森林 model:
Arch = as.data.table(instance$archive, exclude_columns = NULL) # so we keep the uhash
best_RF = Arch[branch.selection == "lrn_ranger"]
best_RF = best_RF[which.min(best_RF$regr.rmse), ] # to get the best RF model from the data table
instance$archive$learner_param_vals(uhash = best_RF$uhash)
lrn_2 = as_learner(graph)
lrn_2$param_set$values = instance$archive$learner_param_vals(uhash = best_RF$uhash)
#lrn_2$param_set$values = instance$archive$learner_param_vals(i = best_RF$batch_nr)
當我使用uhash
或batch_nr
,我無法檢索最佳隨機森林 model 的超參數。 我總是收到存檔中第一行的param_set
,而uhash
和batch_nr
是正確的:
$slct_1.selector
selector_name(c("T", "RH"))
$missind.which
[1] "missing_train"
$missind.type
[1] "factor"
$missind.affect_columns
selector_invert(selector_type(c("factor", "ordered", "character")))
$encode.method
[1] "treatment"
$encode.affect_columns
selector_type("factor")
$branch.selection
[1] "lrn_ranger"
$slct_2.selector
selector_name(c("T"))
$mutate.mutation
$mutate.mutation$Tra.Trafo
~(T^(2))/(2)
$mutate.delete_originals
[1] FALSE
$xgboost.nthread
[1] 1
$xgboost.verbose
[1] 0
$ranger.min.node.size
[1] 15
$ranger.mtry
[1] 1
$ranger.num.threads
[1] 1
$ranger.num.trees
[1] 26
$ranger.sample.fraction
[1] 0.8735846
當我不僅對instance$result_learner_param_vals
result_learner_param_vals 的 output 感興趣時,有人可以給我一個關於如何達到提取其他超參數的目標的提示嗎?
編輯:
我想澄清一些事情,這也與分支有關。 在閱讀@be_marc 的評論后,我不確定它是否打算像那樣工作。 讓我們使用我發布的畫廊示例作為參考。 我想使用 GraphLearner object 比較不同調整分支的結果。 我已經創建了最終的 model,就像在畫廊示例中一樣,在我的例子中是 xgboost model。 我還想為其他分支創建最終模型以進行基准測試。 問題是,如果我不創建原始 graph_learner 的deep clone
,原始graph_learner
會更改參數branch.selection
的值。 為什么我不能只使用普通的克隆? 為什么一定是深度克隆? 它應該像那樣工作嗎? 很可能我不確定克隆和深度克隆之間有什么區別。
# Reference for cloning https://mlr3.mlr-org.com/reference/Resampling.html
# equivalent to object called graph_learner in mlr3 gallery example
graph_learner$param_set$values$branch.selection # original graph_learner object (reference MLR_gallery in first post)
# individually uncomment for different cases
# --------------------------------------------------------------------------------
#MLR_graph = graph # object graph_learner doesn't keeps its original state
#MLR_graph = graph$clone() # object graph_learner doesn't keeps its original state
MLR_graph = graph$clone(deep = TRUE) # object graph_learner keeps its original state
# --------------------------------------------------------------------------------
MLR_graph$param_set$values$branch.selection # value inherited from original graph
MLR_graph$param_set$values$branch.selection = "lrn_MLR" # change set value to other branch
MLR_graph$param_set$values$branch.selection # changed to branch "lrn_MLR"
MLR_lrn = as_learner(MLR_graph) # create a learner from graph with new set branch
# Here we can see the different behaviours based on if we don't clone, clone or deep clone
# at the end, the original graph_learner is supposed to keep it's original state
graph_learner$param_set$values$branch.selection
MLR_lrn$param_set$values$branch.selection
當我不使用深度克隆時,整體上最好的 model lrn
(跳轉到本文開頭)也會受到影響。 就我而言,它是 xgboost。 lrn_MLR
的參數branch.selection
設置為lrn
:
print(lrn)
<GraphLearner:slct_1.copy.missind.imputer_num.encode.featureunion.branch.nop_1.nop_2.slct_2.nop_3.nop_4.mutate.xgboost.ranger.MLR.unbranch>
* Model: list
* Parameters: slct_1.selector=<Selector>, missind.which=missing_train, missind.type=factor,
missind.affect_columns=<Selector>, encode.method=treatment, encode.affect_columns=<Selector>,
branch.selection=lrn_MLR, slct_2.selector=<Selector>, mutate.mutation=<list>, mutate.delete_originals=FALSE,
xgboost.alpha=1.891, xgboost.eta=0.06144, xgboost.lambda=0.01341, xgboost.max_depth=3, xgboost.nrounds=122,
xgboost.nthread=1, xgboost.verbose=0, ranger.num.threads=1
* Packages: mlr3, mlr3pipelines, stats, mlr3learners, xgboost, ranger
* Predict Types: [response], se, distr
* Feature Types: logical, integer, numeric, character, factor, ordered, POSIXct
* Properties: featureless, hotstart_backward, hotstart_forward, importance, loglik, missings, oob_error,
selected_features, weights
編輯 2:好的,我剛剛發現,當我在實驗中與不同的、不同的學習者一起工作時,我應該始終使用深度克隆: https://github.com/mlr-org/mlr3/issues/344
該行為是有意的。
我們修復了最新開發版本 (10.09.2022) 中的錯誤。 你可以安裝它
remotes::install_github("mlr-org/mlr3")
學習者沒有正確地重新組裝。 這再次有效
library(mlr3pipelines)
library(mlr3tuning)
learner = po("subsample") %>>% lrn("classif.rpart", cp = to_tune(0.1, 1))
# hyperparameter tuning on the pima indians diabetes data set
instance = tune(
method = "random_search",
task = tsk("pima"),
learner = learner,
resampling = rsmp("cv", folds = 3),
measure = msr("classif.ce"),
term_evals = 10
)
instance$archive$learner_param_vals(i = 1)
instance$archive$learner_param_vals(i = 2)
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.