如何在 R 中的 elasticnet 中使用不同的 set.seed 获得相同的系数？

Question

This question was closed at CrossValidation as it focussed on programming, so it is more suitable here:

I am running an elastic-net logistic regression on my data. I have looked into how to get replicable coefficients every time I run the same model on the same data. But, it does not seem to happen. I have tried to set nfolds and foldid but once I change the set.seed the coefficients change.

I understand, how the cross-validation works and how the set.seed potentially change the whole ting. Some have suggested setting the foldid as done in my code, but it does not help in my case once the set.seed changes.

What are the possibilities of getting the same coefficients for every run or statistically sound measure for the model coefficients?

df <- read_csv("data.csv")
View(df)

set.seed(123)
library(caret)
library(tidyverse)
library(glmnet)
library(ROCR)
library(doParallel)
registerDoParallel(4, cores = 4)
training.samples <- df$V1 %>% createDataPartition(p = 0.8, list = FALSE)
train <- df[training.samples, ]
test <- df[-training.samples, ]
x.train <- data.frame(train[, names(train) != "V1"])
x.train <- data.matrix(x.train)
y.train <- train$V1
x.test <- data.frame(test[, names(test) != "V1"])
x.test <- data.matrix(x.test)
y.test <- test$V1
foldid <- sample(rep(seq(10), length.out = nrow(train)))
 
list.of.fits <- list()
for (i in 0:10){
    fit.name <- paste0("alpha", i/10) 
    list.of.fits[[fit.name]] <- cv.glmnet(x.train, y.train, type.measure = "dev",
    alpha = i/10, family = "binomial", nfolds = 10, foldid = foldid, parallel = TRUE)
}
coef <- coef(list.of.fits[[fit.name]], s = list.of.fits[[fit.name]]$lambda.1se)
coef

My output ends up like this:

set.seed(123)

(Intercept) -18.533050
V2          -0.0049142
V3          -0.0013228
V4          -0.0029664
V5           0.0123987
V6           0.1433817
V7           .           
V8          -0.0188888
V9           0.0007504
V10         -0.0626482
set.seed(42)

(Intercept) -22.16271709
V2          -0.005898701
V3          -0.001332854
V4          -0.003506514
V5           0.013343484
V6           0.097911065
V7          -0.269346185
V8          -0.024876785
V9           0.027937690
V10         -0.070759818

Answer 1

There are two different type of reproducibility. One is setting the seed, so that one can get the same output for cv.glmnet. And this is what you have done, setting the seed, and providing the folds as input. So if you run the code again with the same seed, it would give you the same results.

In your question and example, you are running it with different cross-validation folds and extracting the coefficient with the 1se. The errors for each lambda value in each sampling will of course differ, which is the purpose of cross-validation, but you would not expect the lambdas to be too different.

We can look at this with an example dataset:

library(glmnet)
library(mlbench)
data(Sonar)
set.seed(111)
idx = sample(nrow(Sonar),150)
x.train = as.matrix(Sonar[idx,1:60])
y.train = as.numeric(Sonar$Class)[idx]
x.test = as.matrix(Sonar[-idx,1:60])
y.test = as.numeric(Sonar$Class)[-idx]

Use 5 different seeds, 1 constant alpha:

o = lapply(1:5,function(i){
set.seed(i)
foldid <- sample(rep(seq(10), length.out = nrow(x.train)))
fit = cv.glmnet(x.train, y.train, type.measure = "dev",
         alpha = 0.5, nfolds = 10, family="binomial",foldid = foldid)

wh = which(fit$lambda==fit$lambda.1se)
data.frame(seed=i,lambda = fit$lambda.1se,error = fit$cvm[wh],
           hi = fit$cvup[wh] , lo = fit$cvlo[wh])
})

See that the lambda 1 se does not differ so much, and also the error:

do.call(rbind,o)
  seed     lambda    error       hi       lo
1    1 0.08047217 1.020349 1.071294 0.969404
2    2 0.14062741 1.099030 1.148053 1.050007
3    3 0.12813445 1.101091 1.160062 1.042121
4    4 0.11675134 1.104327 1.165262 1.043392
5    5 0.10637948 1.059930 1.114897 1.004962

If your data is large enough, you will see these lambda and error values getting closer. So using one of these lambdas should be good enough to give you a prediction that would minimize the error.