如何通过Flask应用程序在Python中使用GDAL打开远程文件

Question

So, I'm developing a Flask application which uses the GDAL library, where I want to stream a .tif file through an url.

Right now I have method that reads a .tif file using gdal.Open(filepath). When run outside of the Flask environment (like in a Python console), it works fine by both specifying the filepath to a local file and a url.

from gdalconst import GA_ReadOnly
import gdal
filename = 'http://xxxxxxx.blob.core.windows.net/dsm/DSM_1km_6349_614.tif'
dataset = gdal.Open(filename, GA_ReadOnly )
if dataset is not None:
    print 'Driver: ', dataset.GetDriver().ShortName,'/', \
      dataset.GetDriver().LongName

However, when the following code is executed inside the Flask environement, I get the following message: ERROR 4: ` http://xxxxxxx.blob.core.windows.net/dsm/DSM_1km_6349_614.tif ' does not exist in the file system, and is not recognised as a supported dataset name.

If I instead download the file to the local filesystem of the Flask app, and insert the path to the file, like this:

block_blob_service = get_blobservice() #Initialize block service
block_blob_service.get_blob_to_path('dsm', blobname, filename) # Get blob to local filesystem, path to file saved in filename
dataset = gdal.Open(filename, GA_ReadOnly)

That works just fine... The thing is, since I'm requesting some big files (200 mb), I want to stream the files using the url instead of the local file reference.

Does anyone have an idea of what could be causing this? I also tried putting "/vsicurl_streaming/" in front of the url as suggested elsewhere.

I'm using Python 2.7, 32-bit with GDAL 2.0.2

Answer 1

Please try the follow code snippet:

from gzip import GzipFile
from io import BytesIO
import urllib2
from uuid import uuid4
from gdalconst import GA_ReadOnly
import gdal

def open_http_query(url):
    try:
        request = urllib2.Request(url, 
            headers={"Accept-Encoding": "gzip"})
        response = urllib2.urlopen(request, timeout=30)
        if response.info().get('Content-Encoding') == 'gzip':
            return GzipFile(fileobj=BytesIO(response.read()))
        else:
            return response
    except urllib2.URLError:
        return None


url = 'http://xxx.blob.core.windows.net/container/example.tif'
image_data = open_http_query(url)
mmap_name = "/vsimem/"+uuid4().get_hex()
gdal.FileFromMemBuffer(mmap_name, image_data.read())
dataset = gdal.Open(mmap_name)
if dataset is not None:
    print 'Driver: ', dataset.GetDriver().ShortName,'/', \
      dataset.GetDriver().LongName

Which use a GDAL memory-mapped file to open an image retrieved via HTTP directly as a NumPy array without saving to a temporary file. Refer to https://gist.github.com/jleinonen/5781308 for more info.