R CMD check fails with ubuntu when trying to download file, but function works within R

Question

I am writing an R package and one of its functions download and unzips a file from a link (it is not exported to the user, though):

download_f <- function(download_dir) {
  utils::download.file(
    url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php",
    destfile = file.path(download_dir, "fines.rar"),
    mode = 'wb',
    method = 'libcurl'
  )

  utils::unzip(
    zipfile = file.path(download_dir, "fines.rar"),
    exdir = file.path(download_dir)
  )
}

This function works fine with me when I run it within some other function to compile an example in a vignette.

However, with R CMD check in github action, it fails consistently on ubuntu 16.04, release and devel. It [says][1]:

Error: Error: processing vignette 'IBAMA.Rmd' failed with diagnostics:
cannot open URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php'
--- failed re-building ‘IBAMA.Rmd’

SUMMARY: processing the following file failed:
  ‘IBAMA.Rmd’

Error: Error: Vignette re-building failed.
Execution halted
Error: Error in proc$get_built_file() : Build process failed
Calls: <Anonymous> ... build_package -> with_envvar -> force -> <Anonymous>
Execution halted
Error: Process completed with exit code 1.

When I run devtools::check() it never finishes running it, staying in "creating vignettes" forever. I don't know if these problems are related though because there are other vignettes on the package.

I pass the R CMD checks with mac os and windows. I've tried switching the "mode" and "method" arguments on utils::download.file , but to no avail.

Any suggestions? [1]: https://github.com/datazoompuc/datazoom.amazonia/pull/16/checks?check_run_id=2026865974

Answer 1

The download fails because libcurl tries to verify the webservers certificate, but can't.

I can reproduce this on my system:

trying URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php'
Error in utils::download.file(url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php",  : 
  cannot open URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php'
In addition: Warning message:
In utils::download.file(url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php",  :
  URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php': status was 'SSL peer certificate or SSH remote key was not OK'

The server does not allow you to download from http but redirects to https, so the only thing to do now is to tell libcurl to not check the certificate and accept what it is getting.

You can do this by specifying the argument -k to curl

download_f <- function(download_dir) {
  utils::download.file(
    url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php",
    destfile = file.path(download_dir, "fines.rar"),
    mode = 'wb',
    method = 'curl',
    extra = '-k'
  ) 
  
  utils::unzip(
    zipfile = file.path(download_dir, "fines.rar"),
    exdir = file.path(download_dir)
  ) 
} 

This also produces some download progress bar, you can silence this by setting extra to -k -s

This now opens you up to a Machine In The Middle Attack. (You possibly already are attacked this way, there is no way to check without verifying the current certificate with someone you know at the other side) So you could implement an extra check, eg check the sha256sum of the downloaded file and see if it matches what you expect to receive before proceeding.

myfile <- system.file("fines.rar")
hash <- sha256(file(myfile))