capturing convergence message from lme4 package in R

Question

I was wondering if there is a way to write a logical test ( TRUE/FALSE ) to show whether a model from lme4 package has converged or not?

An example is shown below, I want to capture if any model comes with the convergence warning (ie, Model failed to converge ) message?

library(lme4)

dat <- read.csv('https://raw.githubusercontent.com/rnorouzian/e/master/nc.csv')

m <- lmer(math ~ ses*sector + (ses | sch.id), data = dat)

Warning message:
In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
  Model failed to converge with max|grad| = 0.00279 (tol = 0.002, component 1)

Answer 1

We can use tryCatch , using withCallingHandlers taking inspiration from this post.

dat <- read.csv('https://raw.githubusercontent.com/rnorouzian/e/master/nc.csv')

m <- tryCatch({
          withCallingHandlers({
            error <- FALSE
            list(model = lmer(math ~ ses*sector + (ses | sch.id), data = dat),
                 error = error)
          },warning = function(w) {
              if(grepl('failed to converge', w$message)) error <<- TRUE
          }
          )})


m$model
#Linear mixed model fit by REML ['lmerMod']
#Formula: math ~ ses * sector + (ses | sch.id)
#   Data: dat
#REML criterion at convergence: 37509.07
#Random effects:
# Groups   Name        Std.Dev. Corr
# sch.id   (Intercept) 1.9053       
#          ses         0.8577   0.46
# Residual             3.1930       
#Number of obs: 7185, groups:  sch.id, 160
#Fixed Effects:
#(Intercept)          ses       sector   ses:sector  
#     11.902        2.399        1.677       -1.322  
#convergence code 0; 0 optimizer warnings; 1 lme4 warnings 

m$error
#[1] TRUE

The output m is a list with model and error elements.

---

If we need to test for warning after the model has been created we can use :

is_warning_generated <- function(m) {
  df <- summary(m)
  !is.null(df$optinfo$conv$lme4$messages) && 
           grepl('failed to converge', df$optinfo$conv$lme4$messages)
}

m <- lmer(math ~ ses*sector + (ses | sch.id), data = dat)
is_warning_generated(m)
#[1] TRUE

Answer 2

We can use safely from purrr . It will also return the error as a list element and captures the error. If there are no error, it will be NULL

library(purrr)
safelmer <- safely(lmer, otherwise = NA)
out <- safelmer(math ~ ses*sector + (ses | sch.id), data = dat)

Answer 3

I applied Ronak's solution to my own simulation data and found a problem. The message may be a vector of multiple entries, leading also grepl() to have multiple entries. However, the && operator compares the string only to the first entry, such that further occurrences of 'failed to converge' are unobserved. To avoid this behavior, I changed && to &. Now a problem occurred if there was no message at all. In this case the !is.null() part becomes correctly FALSE (ie, no warning generated), but the grepl() part becomes logical(0) and the function value becomes FALSE & logical(0) which is logical(0). In fact it would work for FALSE && logical(0) which is FALSE (correct).

A solution that worked for me is if(is.null(mess)) FALSE else grepl('failed to converge', mess)

which in case of a failure to converge provides a vector with a TRUE at the entry where the warning was placed. This vector may be evaluated, for example, by building the numeric (or Boolean) sum which becomes greater 0 or TRUE.

Answer 4

I'm just going to say that @RonakShah's is_warning_generated could be made slightly more compact:

function(m) { 
    w <- m@optinfo$conv$lme4$messages
    !is.null(w) && grepl('failed to converge', w) 
}

Answer 5

> sm=summary(model)
> sm$optinfo$conv$lme4$messages
[1] "Model failed to converge with max|grad| = 0.0120186 (tol = 0.002, component 1)"
>