在R中下载文件时捕获FTP日志

Question

I want to download a file via ftp, and also capture the ftp log. I managed to get this far:

options(internet.info = 0)  # displays the ftp log in the console

x <- capture.output(
  download.file("ftp://speedtest.tele2.net/1KB.zip", destfile = tempfile()),
  type = "message"
)

This displays the ftp log in the console and captures some download.file() output to the variable x , but it does not capture the ftp log. I also tried some fiddling around with sink() , but no success.

Can anyone explain to me why this is not working, and is there any way to achieve what I want? I want to process some ftp messages so simply getting back an error code would not suffice.

Answer 1

If you're using Linux or Cygwin or Git bash, you can pipe the results of a script to a text file. If you put your code (without the output capturing) into a file called script.R you can pipe all the output to a text file:

Rscript script.R &> script.txt

You can do something like this within R - but it feels evil to me:

uri <- "ftp://speedtest.tele2.net/1KB.zip"
destination <- tempfile()

system2("Rscript", 
        c("-e", 
          sprintf("\"options(internet.info = 0);download.file('%s', destfile = '%s', quiet = FALSE)\"", 
                  uri, 
                  destination)), 
        stderr = TRUE, stdout = TRUE)

Answer 2

With the help of sebastian-c and the original package author I was able to devise a solution for my problem with the curl package. I'll try to preserve it here for posterity:

curl_download_with_log <- function(
  url,
  destfile
){
  log <- rawConnection(raw(), 'r+')
  on.exit(close(log))
  stopifnot(is.character(url))

  h <- curl::new_handle(
    debugfunction = function(type, data){
      if(type %in% c(0, 1, 2)){ 
        writeBin(data, log)
      }
    },
    verbose = TRUE
  )

  try({
    curl::curl_download(url, destfile = destfile, handle = h)
  })

  rawToChar(rawConnectionValue(log))
}

This function will save the file to destfile , and return the log as a character vector.

Explaination:

• debugfunction tells libcurl what to do with the debugging data. In this case, write it to a binary connection

• if(type %in% c(0, 1, 2)){...} ensures that only log data is written to the debug connection. If you leave this out, the whole file you download gets written to the debug log along with the protocol data (see link for explaination of the different debug levels)