Quantmod R :specifyModel() Error: NG3(xts object) download failed after two attempts

Question

 > sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 18.3

Matrix products: default
BLAS: /usr/lib/libblas/libblas.so.3.6.0
LAPACK: /usr/lib/lapack/liblapack.so.3.6.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] lubridate_1.7.4 quantmod_0.4-13 TTR_0.23-4      xts_0.11-2     
[5] zoo_1.8-4      

loaded via a namespace (and not attached):
 [1] compiler_3.5.1  magrittr_1.5    tools_3.5.1     curl_3.2       
 [5] yaml_2.2.0      Rcpp_1.0.0      stringi_1.2.4   grid_3.5.1     
 [9] stringr_1.3.1   lattice_0.20-38
> 

I run this code:

 Lesson4 <- function()
 {
  # Dec 9 2018 - next steps to learn in order to 
 # perform backtest
 #steps .5?? specifymodel() function
 #steps 1 buildmodel() function
 #steps 2 trademodel() function
 #steps 3 call functions to get model performance 
 #details

library("lubridate")
library("quantmod")
library(xts)
setwd("/home/gabe/Learn")

NG2 <- read.csv("NGdec18.csv",
               stringsAsFactors = FALSE,
               header = FALSE)

dates <- as.character(NG2$V1)
dates2 <- as.POSIXct(dates, format = "%Y%m%d %H:%M:%S")
NG2[, 1] <- dates2

   names(NG2)[2]  <- paste("NG3.Open")
   names(NG2)[3]  <- paste("NG3.High")
   names(NG2)[4]  <- paste("NG3.Low")
   names(NG2)[5]  <- paste("NG3.Close")
   names(NG2)[6]  <- paste("NG3.Volume")
   NG2 <- subset(NG2,  NG2$V1 > "2018-10-01")

   NG3 <- xts(NG2[-1], order.by = NG2[, 1])

   print(c("class=", class(NG3)))
   print(NG3[1:2,])
   print(head(OpHi(NG3)))
   print(head(OpCl(NG3)))
   specifyModel(Next(OpCl(NG3)) ~ OpHi(NG3))


 }

It works fine when I use same code with GE stock prices from yahoo...but when I use this intraday data from a csv file it fails. However i checked and the data "NG3" is a xts file. I also carefully named the columns because that was a problem. I'm also using the simplest little model I can think of and it seems each of the parts of the silly model are legit objects. You can see that in the output:

> Lesson4()
[1] "class=" "xts"    "zoo"   

 NG3.Open NG3.High NG3.Low NG3.Close NG3.Volume
2018-10-01 00:10:00    3.104    3.107   3.104     3.107         17
2018-10-01 00:20:00    3.107    3.107   3.107     3.107          5
                        OpHi.NG3
      2018-10-01 00:10:00 0.0009664948
      2018-10-01 00:20:00 0.0000000000
      2018-10-01 00:30:00 0.0000000000
      2018-10-01 00:40:00 0.0000000000
      2018-10-01 00:50:00 0.0000000000
       2018-10-01 01:00:00 0.0003226847
                         OpCl.NG3
       2018-10-01 00:10:00  0.0009664948
       2018-10-01 00:20:00  0.0000000000
       2018-10-01 00:30:00 -0.0016097875
       2018-10-01 00:40:00  0.0000000000
       2018-10-01 00:50:00  0.0000000000
       2018-10-01 01:00:00  0.0003226847
    Warning: NG3 download failed; trying again.
     Show Traceback

     Rerun with Debug
     Error: NG3 download failed after two attempts. Error 
    message:
    HTTP error 404. 

I don't even need internet connection for this...why is it telling me HTTP error?more importantly...can anyone help me get this specifymodel() function to work without this error?

here is the traceback:

9.
stop(Symbols.name, " download failed after two attempts. Error", 
    " message:\n", attr(dl, "condition")$message, call. = FALSE) 
8.
getSymbols.yahoo(Symbols = "NG3", env = <environment>, verbose = FALSE, 
    warnings = TRUE, auto.assign = TRUE) 
7.
do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
    env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
    ...)) 
6.
getSymbols(V, env = env) 
5.
FUN(X[[i]], ...) 
4.
lapply(vars, function(V) {
    if (!exists(V)) {
        getSymbols(V, env = env)
    } ... 
3.
getModelData(new.quantmod, na.rm = na.rm) 
2.
specifyModel(Next(OpCl(NG3)) ~ OpHi(NG3)) at ghtest.R#170
1.
Lesson4() 

also here is the head() of the original csv file NGdec18.csv:

                 V1    V2    V3    V4    V5  V6
1 20180924  18:00:00 3.102 3.106 3.102 3.104 118
2 20180924  18:10:00 3.102 3.103 3.101 3.103   6
3 20180924  18:20:00 3.103 3.103 3.103 3.103   0
4 20180924  18:30:00 3.103 3.103 3.103 3.103   0
5 20180924  18:40:00 3.103 3.103 3.103 3.103   0
6 20180924  18:50:00 3.105 3.105 3.105 3.105   1

csv file can be downloaded here: http://www.sharecsv.com/s/53f6728024686c49d2fc81273a7f7464/NGdec18.csv

  Lesson4 <- function()
 {

     library("lubridate")
     library("quantmod")
     library(xts)
     setwd("/home/gabe/Learn")

     NG2 <- read.csv("NGdec18.csv",
                     stringsAsFactors = FALSE,
                     header = FALSE)

     dates <- as.character(NG2$V1)
     dates2 <- as.POSIXct(dates, format = "%Y%m%d %H:%M:%S")
     NG2[, 1] <- dates2

     names(NG2)[2]  <- paste("NG3.Open")
     names(NG2)[3]  <- paste("NG3.High")
     names(NG2)[4]  <- paste("NG3.Low")
     names(NG2)[5]  <- paste("NG3.Close")
     names(NG2)[6]  <- paste("NG3.Volume")
  #  setSymbolLookup("symbol.lookup" = FALSE)
    # options("getSymbols.sources" = "csv")
     setSymbolLookup(NG3='csv')
     NG3 <- xts(NG2[-1], order.by = NG2[, 1])

     print(c("class=", class(NG3)))
     print(NG3[1:2,])
     print(head(OpHi(NG3)))
     print(head(OpCl(NG3)))
     specifyModel(Next(OpCl(NG3)) ~ OpHi(NG3))
 }

I tried your suggestions and one of my own...and it got rid of the 'yahoo' source problem, but still get this error:

Error in `dimnames<-.xts`(`*tmp*`, value = dn) : 
 length of 'dimnames' [2] not equal to array extent 

with traceback:

11.
`dimnames<-.xts`(`*tmp*`, value = dn) 
10.
`dimnames<-`(`*tmp*`, value = dn) 
9.
`colnames<-`(`*tmp*`, value = paste(toupper(gsub("\\^", "", Symbols[[i]])), 
    col.names, sep = ".")) 
8.
getSymbols.csv(Symbols = "NG3", env = <environment>, verbose = FALSE, 
    warnings = TRUE, auto.assign = TRUE) 
7.
do.call(paste("getSymbols.", symbol.source, sep = ""), list(Symbols = current.symbols, 
    env = env, verbose = verbose, warnings = warnings, auto.assign = auto.assign, 
    ...)) 
6.
getSymbols(V, env = env) 
5.
FUN(X[[i]], ...) 
4.
lapply(vars, function(V) {
    if (!exists(V)) {
        getSymbols(V, env = env)
    } ... 
3.
getModelData(new.quantmod, na.rm = na.rm) 
2.
specifyModel(Next(OpCl(NG3)) ~ OpHi(NG3)) at ghtest.R#166
1.
Lesson4() 

Answer 1

This is a guess since your code runs without error on my Linux box running R 3.5.1 and here's the OS and packages part of sessioninfo():

sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.1 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3
LAPACK: /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so
   #omitted locales
other attached packages:
[1] lubridate_1.7.4 quantmod_0.4-13 TTR_0.23-4      xts_0.11-2      zoo_1.8-4      

loaded via a namespace (and not attached):
 [1] compiler_3.5.1  magrittr_1.5    tools_3.5.1     curl_3.2        yaml_2.2.0     
 [6] Rcpp_1.0.0      stringi_1.2.4   grid_3.5.1      stringr_1.3.1   lattice_0.20-38

So my guess is that you need to set the quantmod option for symbol.lookup to FALSE. I cannot test my theory because my system succeeds with your code but you could try:

setSymbolLookup("symbol.lookup" = FALSE)

The goal is to get teh symbol lookup default to NOT use "yahoo". At least it didn't cause an errors or break your Lesson4 code. Admittedly it wasn't a very interesting result:

quantmod object:        Build date:   

Model Specified: 
     Next(OpCl(NG3)) ~ OpHi(NG3) 

Model Target:  Next.OpCl.NG3         Product:  NG3 
Model Inputs:   

Fitted Model: 

    None Fitted

After looking at the code for getSymbols I have yet another idea:

options("getSymbols.sources" = "csv")
getOption("getSymbols.sources")
[1] "csv"