Error when using calibrate function for ols model fitted in rms package

Question

I am attempting to use the rms package in R to evaluate the predictive accuracy of a linear model but am having difficulties with producing a calibration plot for some reason.

Specifically, I receive the error message: "Error in !fail : invalid argument type" when using calibrate()

Here is a simple reproducible example to demonstrate my problem:

> library(rms)
> library(MASS)
> data(whiteside)
> w.Before <- whiteside[whiteside$Insul=="Before",]
# For comparability with an example in MASS.
> dd <- datadist(w.Before)
> options(datadist="dd")

> mod1 <- ols(Gas ~ Temp,data=w.Before,
                         x=TRUE,y=TRUE)
> mod1
            Coef    S.E.   t      Pr(>|t|)
 Intercept  6.8538 0.1184  57.88 <0.0001 
 Temp      -0.3932 0.0196 -20.08 <0.0001

# ols() estimates the coefficients correctly

# Five-fold cross-validation for this model fit:
> validate(mod1,bw=FALSE,method="crossvalidation",B=5)
           index.orig training    test optimism index.corrected n
 R-square      0.9438   0.9431  0.8460   0.0971          0.8467 5
 MSE           0.0731   0.0709  0.0929  -0.0220          0.0951 5
 g             1.2791   1.2664  1.2222   0.0442          1.2349 5
 Intercept     0.0000   0.0000 -0.1661   0.1661         -0.1661 5
 Slope         1.0000   1.0000  1.0360  -0.0360          1.0360 5

# try using default argument options:
> calibrate(mod1)
Error in !fail : invalid argument type

# try using some arguments specific to the fitted ols object:
> calibrate(mod1,predy=median(w.Before$Gas),method="crossvalidation",B=5)
Error in !fail : invalid argument type

I have done some rudimentary debugging (below) - perhaps this might provide a clue?

mod1 <- ols(Gas ~ Temp,data=w.Before,
                         x=TRUE,y=TRUE)

# Switch on debug argument:
> calibrate(mod1, debug = TRUE)

Subscripts of training sample:
 [1]  3  9 15 22 21  4  7 22  8  5 11 10 25  4 10  5 18 23  6  1 19 10  9  6 22 25

Subscripts of test sample:
 [1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

Error in !fail : invalid argument type
> traceback()
3: predab.resample(fit, method = method, fit = fitit, measure = cal.error, 
   pr = pr, B = B, bw = bw, rule = rule, type = type, sls = sls, 
   aics = aics, force = force, estimates = estimates, non.slopes.in.x = 
model == 
       "ol", smoother = smoother, predy = predy, model = model, 
   kint = kint, penalty.matrix = penalty.matrix, ...)
2: calibrate.default(mod1, debug = TRUE)
1: calibrate(mod1, debug = TRUE)

> options(error = recover)
> calibrate(mod1)
Error in !fail : invalid argument type

Enter a frame number, or 0 to exit   

1: calibrate(mod1)
2: calibrate.default(mod1)
3: predab.resample(fit, method = method, fit = fitit, measure = cal.error, pr = p

Selection: 3
Called from: calibrate.default(mod1)

# Entering object names in predab.resample function from last object in this function until I find an error / missing or suspect values:
Browse[1]> res                      
Error during wrapup: object 'res' not found

Browse[1]> varin                                                            
Error during wrapup: object 'varin' not found

Browse[1]> index.orig
 [1] -0.0146323 -0.0095743 -0.0052462 -0.0023136  0.0004922  0.0032980  0.0061038
 [8]  0.0081017  0.0111468  0.0147536  0.0183605  0.0219674  0.0255743  0.0291812
 [15]  0.0346582  0.0427575  0.0513092  0.0584638  0.0626379  0.0668120  0.0709861
 [22]  0.0751603  0.0793344  0.0832308  0.0830547  0.0828785  0.0827024  0.08 25263
 [29]  0.0823502  0.0821740  0.0819979  0.0818218  0.0816456  0.0814695  0.0812934
 [36]  0.0820341  0.0833151  0.0845299  0.0857447  0.0858007  0.0842924  0.0827842
 [43]  0.0801835  0.0750078  0.0698321  0.0646565  0.0591428  0.0513195  0.0434962
 [50]  0.0356728
attr(,"keepinfo")
attr(,"keepinfo")$orig.cal
 [1] 3.851 3.893 3.935 3.976 4.017 4.057 4.098 4.137 4.178 4.219 4.261 4.302 4.343
 [14] 4.385 4.428 4.474 4.520 4.565 4.607 4.649 4.691 4.732 4.774 4.816 4.854 4.891
 [27] 4.929 4.966 5.004 5.041 5.079 5.116 5.154 5.191 5.229 5.267 5.306 5.345 5.384
[40] 5.422 5.458 5.494 5.530 5.562 5.595 5.627 5.659 5.689 5.719 5.749

Browse[1]> optimism      
Error during wrapup: object 'optimism' not found

Hopefully this provides some clue. Can someone please help me to resolve this error, and ideally, obtain the calibration plot for this ols model?

Thanks, in advance.

Ross.

Answer 1

I was getting the same error and after poking around on Github, discovered that it's a known issue.

https://github.com/harrelfe/rms/issues/61

Looks like there was a commit that fixes the issue back in April, but an updated package hasn't yet been published to CRAN.

https://github.com/harrelfe/rms/commit/6bcaee45455c0e0c4ec163cd1ac325010f7648fa