如何为stepAIC添加特定条件

Question

I am running a regression with 37 variables, and I am using stepAIC to perform model selection. I do NOT want a predictive model. I just want to find out what varibles have the best explanatory power.

My current code looks like:

fitObject <- lm(mydata)
DEP.select <- stepAIC(fitObject, direction = 'both', scope= list(lower = ~AUC), trace = F, k = log(obs))
# DEP is my dependent variable, and AUC is an independent variable I was want to have in my model.

The problem is that a lot of my variables have high correlation, and the result stepAIC gives me contains several of those highly correlated variables. Notice that I have forced AUC in the model, multicollinearity is a problem especially when those variables highly correlated with AUC are chosen in the model.

Is there a way to specify in the function some thresholds for correlation or p-value of the coefficients?

Or any comments on other approaches that can solve my problem are welcome.

Thank you!

Answer 1

Perhaps Variance Inflation Factor will work better for you. This article explains some of the logic. http://en.wikipedia.org/wiki/Variance_inflation_factor

Example use:

v=ezvif(df,yvar ='columnNameOfWhichYouAreTryingToPredict')

Here is the function I wrote that combines VIF::vif with cross validation.

require(VIF)
require(cvTools);
#returns selected variables using VIF and kfolds cross validation 
ezvif=function(df,yvar,folds=5,trace=F){
  f=cvFolds(nrow(df),K=folds);
  findings=list();
  for(v in names(df)){
    if(v==yvar)next;
    findings[[v]]=0; 
  }
  for(i in 1:folds){   
    rows=f$subsets[f$which!=i]
    y=df[rows,yvar];
    xdf=df[rows,names(df) != yvar]; #remove output var    
    vifResult=vif(y,xdf,trace=trace,subsize=min(200,floor(nrow(xdf))))
    for(v in names(xdf)[vifResult$select]){
      findings[[v]]=findings[[v]]+1; #vote
    }
  }
  findings=(sort(unlist(findings),decreasing = T))    
  if(trace) print(findings[findings>0]); 
  return( c(yvar,names(findings[findings==findings[1]])) )  
}

Answer 2

I would recommend to remove the variables with high correlations. The libraries caret and corrplot can help:

library(corrplot)
library(caret)
dm = data.matrix(mydata[,names(mydata) != 'DEP'] #without your outcome var

Visualize your correlations clustering highly correlated together

corrplot(cor(dm), order = 'hclust')

And find the indices of variables that you could remove due to high (>0.75) correlations

findCorrelations(cor(dm), 0.75)

Removing these variables can improve your model. After removing the variables, continue doing the stepAIC as you described in your question.

Answer 3

To assess multicollinearity between predictors when running the dredge function (MuMIn package), include the following max.r function as the "extra" argument:

max.r <- function(x){
  corm <- cov2cor(vcov(x))
  corm <- as.matrix(corm)
  if (length(corm)==1){
    corm <- 0
    max(abs(corm))
  } else if (length(corm)==4){
  cormf <- corm[2:nrow(corm),2:ncol(corm)]
  cormf <- 0
  max(abs(cormf))
  } else {
    cormf <- corm[2:nrow(corm),2:ncol(corm)]
    diag(cormf) <- 0
    max(abs(cormf))
  }
}

then simply run dredge specifying the number of predictor variables and including the max.r function:

options(na.action = na.fail)
Allmodels <- dredge(Fullmodel, rank = "AIC", m.lim=c(0, 3), extra= max.r) 
Allmodels[Allmodels$max.r<=0.6, ] ##Subset models with max.r <=0.6 (not collinear)
NCM <- get.models(Allmodels, subset = max.r<=0.6) ##Retrieve models with max.r <=0.6 (not collinear)
model.sel(NCM) ##Final model selection table

This works for lme4 models. For nlme models see: https://github.com/rojaff/dredge_mc