從 Tidymodels 中的擬合工作流中獲取訓練數據的 AUC？

Question

我正在努力解決如何使用 tidymodels 從邏輯回歸 model 獲得 AUC。

這是一個使用內置mpg數據集的示例。

library(tidymodels)
library(tidyverse)

# Use mpg dataset
df <- mpg

# Create an indicator variable for class="suv"
df$is_suv <- as.factor(df$class == "suv")

# Create the split object
df_split <- initial_split(df, prop=1/2)

# Create the training and testing sets
df_train <- training(df_split)
df_test <- testing(df_split)

# Create workflow
rec <-
  recipe(is_suv ~ cty + hwy + cyl, data=df_train)

glm_spec <-
  logistic_reg() %>%
  set_engine(engine = "glm")

glm_wflow <- 
  workflow() %>%
  add_recipe(rec) %>%
  add_model(glm_spec)

# Fit the model
model1 <- fit(glm_wflow, df_train)

# Attach predictions to training dataset
training_results <- bind_cols(df_train, predict(model1, df_train))

# Calculate accuracy
accuracy(training_results, truth = is_suv, estimate = .pred_class)

# Calculate AUC??
roc_auc(training_results, truth = is_suv, estimate = .pred_class)

最后一行返回此錯誤：

> roc_auc(training_results, truth = is_suv, estimate = .pred_class)
Error in metric_summarizer(metric_nm = "roc_auc", metric_fn = roc_auc_vec,  : 
  formal argument "estimate" matched by multiple actual arguments

Answer 1

由於您正在執行二進制分類， roc_auc()期望與“相關”class 相對應的 class 概率向量，而不是預測的 class。

您可以使用predict(model1, df_train, type = "prob")獲得此信息。 或者，如果您使用的是 0.2.2 或更高版本的工作流，則可以使用augment()來獲取 class 預測和概率，而無需使用bind_cols() 。

library(tidymodels)
library(tidyverse)

# Use mpg dataset
df <- mpg

# Create an indicator variable for class="suv"
df$is_suv <- as.factor(df$class == "suv")

# Create the split object
df_split <- initial_split(df, prop=1/2)

# Create the training and testing sets
df_train <- training(df_split)
df_test <- testing(df_split)

# Create workflow
rec <-
  recipe(is_suv ~ cty + hwy + cyl, data=df_train)

glm_spec <-
  logistic_reg() %>%
  set_engine(engine = "glm")

glm_wflow <- 
  workflow() %>%
  add_recipe(rec) %>%
  add_model(glm_spec)

# Fit the model
model1 <- fit(glm_wflow, df_train)

# Attach predictions to training dataset
training_results <- augment(model1, df_train)

# Calculate accuracy
accuracy(training_results, truth = is_suv, estimate = .pred_class)
#> # A tibble: 1 x 3
#>   .metric  .estimator .estimate
#>   <chr>    <chr>          <dbl>
#> 1 accuracy binary         0.795

# Calculate AUC
roc_auc(training_results, truth = is_suv, estimate = .pred_FALSE)
#> # A tibble: 1 x 3
#>   .metric .estimator .estimate
#>   <chr>   <chr>          <dbl>
#> 1 roc_auc binary         0.879

^{由代表 package (v1.0.0) 於 2021 年 4 月 12 日創建}