在R中使用卓越的预测能力（SPA）测试

Question

Does anynoe know hot to use the SPA test in R/Matlab or other software; it is a statistical method to evaluate models. I knew that there is a R package called "ttrTests" has a relevant SPA function, but it looks like suitable for comparing portfolio strategies, rather than comparing general models in terms of some loss function. Can someone tell me other source or how to prepare the data suitable for the "ttrTests" package.

Answer 1

The Model Confidence Set package tests multiple models for superior predictive ability.

install.packages("MCS")
library(MCS)
data(Loss)
MCS <- MCSprocedure(Loss=Loss[,1:5],alpha=0.2,B=5000,statistic='Tmax',cl=NULL)

...and the output:

> MCS <- MCSprocedure(Loss=Loss[,1:5],alpha=0.2,B=5000,statistic='Tmax',cl=NULL)

###########################################################################################################################
Superior Set Model created  :
             Rank_M        v_M  MCS_M Rank_R         v_R  MCS_R         Loss
sGARCH-norm       4  0.8201805 0.6034      4  1.43408052 0.3576 0.0004042581
sGARCH-std        5  0.9649670 0.5008      5  3.22834167 0.0058 0.0004010655
sGARCH-ged        1 -1.3942903 1.0000      3  0.21893448 0.9940 0.0003986329
sGARCH-snorm      2 -1.3101987 1.0000      2  0.08452883 0.9998 0.0003982803
sGARCH-sstd       3 -0.4739630 1.0000      1 -0.08452883 1.0000 0.0003977886
p-value :
[1] 0.5008

###########################################################################################################################
> 

Answer 2

You could take a look at the MFE Toolbox for Matlab. It includes two distinct multiple hypotesis tests that may be helpful for you:

• Model Confidence Set ( mcs function)
• Reality Check and Test for Superior Predictive Accuracy ( bsds function)

The framework is pretty well documented and you can see these implementations from this page .