R: read GeoTiff from gz file with a connection and raster package

Question

I would like to read a gzipped GeoTIFF from a server without downloading it. I just don't want to create a lot of temporary files that I have to delete later on.

I see it is possible with .csv.gz files.

With download I do it in the following way:

library(raster)
link <- "ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010
/SRTM_ff03_n000e010.tif.gz"

download.file(link, "test.tif.gz")
gunzip("test.tif.gz")
myras <- raster("test.tif")
plot(myras)

I can read an uncompressed file directly from a link:

link <- "http://download.osgeo.org/geotiff/samples/usgs/o41078a5.tif"
myras <- raster(link)
plot(myras)
myextent <- drawExtent()
plot(myras, ext=myextent)

Here I realize that it might not be a good Idea to not download it to local storage, because I assume that every action you subsequently do with myras needs the data to flow over the internet again. But anyway, just for proof of concept I would like to do it. And there are cases where you just want to display the TIFF without doing any furher calculations with it and therefore don't want to create a temporary file for it.

To read the (downloaded) tiff.gz file without uncompressing it first I tried:

> raster(gzfile("test.tif.gz"))
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"gzfile"’

To read a tiff.gz file directly from the server with a connection I tried the following:

> con <- gzcon(url("ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010/SRTM_ff03_n000e010.tif.gz"))
> raster(con)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"gzcon"’

> raw <- textConnection(readLines(con))
> raster(raw)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"textConnection"’

> rawBin <- textConnection(readBin(con))
Error in readBin(con) : argument "what" is missing, with no default

> con <- gzfile("ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010/SRTM_ff03_n000e010.tif.gz")
> myras <- raster(con)
Error in (function (classes, fdef, mtable)  : 
  unable to find an inherited method for function ‘raster’ for signature ‘"gzfile"’

I found this Stackoverflow question about how to read a zipped binary file connection, but I am not sure whether and GeoTIFF is binary (is it?) and which parameters to pass to the readBin() function.

I feel like randomly trying things out because I don't really understand how connections work. Can anyone help me with this?

Answer 1

I found a solution for this, maybe you have too.

The function readTIFF() from the tiff package can read a tiff image from a raw vector. So you can read your connection into a raw vector with readBin() , then read that raw vector with readTiff() .

# create connection to a gz file
con <- gzfile("test.tif.gz", open = "rb")

# read data from this connection into a raw vector
myras.raw <- readBin(con, what = "raw", n = 1e10)

# read this raw vector
myras <- readTIFF(myras.raw)

I hope this can help :)

Answer 2

Another solution is that the underlying GDAL library provides some specific 'virtual' file systems that allow a zipped and/or remote resource to be read.

https://gdal.org/user/virtual_file_systems.html

The raster command supports those, basically passing the provided file path directly to GDAL to read the data. So you should be able to use the following to load the data directly.

library(raster)
link <- "/vsigzip//vsicurl/ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n000/SRTM_ff03_n000e010
/SRTM_ff03_n000e010.tif.gz"