App Engine Flexible - Docker file fails to install GDAL

Question

I am trying to deploy a Django application to App Engine Flexible Environment. My dockerfile is failing to install GDAL.

This is the error message that i get when running gcloud app deploy :

  File "/env/lib/python3.7/site-packages/django/contrib/gis/gdal/libgdal.py", line 42, in <module>
    % '", "'.join(lib_names)
django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.4.0", "gdal2.3.0", "gdal2.2.0", "gdal2.1.0", "gdal2.0.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.
[2020-04-24 16:12:26 +0000] [8] [INFO] Worker exiting (pid: 8)
[2020-04-24 16:12:26 +0000] [1] [INFO] Shutting down: Master
[2020-04-24 16:12:26 +0000] [1] [INFO] Reason: Worker failed to boot.

This is my dockerfile:

FROM ubuntu:bionic

RUN apt-get update && apt-get install -y \
  binutils \
  gdal-bin \
  python3-gdal \
  ibgdal-dev \
  libproj-dev

# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
# Use -p python3 or -p python3.7 to select python version. Default is version 2.
RUN virtualenv /env -p python3.7



# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
RUN pip install -r requirements.txt

# Add the application source code.
ADD . /


# Run a WSGI server to serve the application. gunicorn must be declared as
# a dependency in requirements.txt.
CMD gunicorn -b :$PORT main:app

and this is my app.yaml:

runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app

runtime_config:
  # You can also specify 2 for Python 2.7
  python_version: 3.7

I am aware that I am asking a very similar question that being answered here , but the askers own solution doesn't seem to be working.

Answer 1

To me, it seems as if you are missing a proper gdal installation. Indeed it can be bit tricky to install but on ubuntu (also with docker ), I generally have good experiences installing it from ubuntugis .

Here's an example:

FROM ubuntu:bionic

RUN apt-get update && apt-get install -y \
    software-properties-common \
    python3 \
    python3-dev \
    python3-pip \
  && rm -rf /var/lib/apt/lists/*

RUN add-apt-repository ppa:ubuntugis/ppa

RUN apt-get update && apt-get install -y \
    gdal-bin=2.4.2+dfsg-1~bionic0 \
    python3-gdal \
  && rm -rf /var/lib/apt/lists/*

RUN apt-get update && apt-get install -y libpq-dev \
  && rm -rf /var/lib/apt/lists/*
...

Answer 2

I think the issue is in the Docker file. You have the following:

...
RUN apt-get update && apt-get install -y \
  binutils \
  gdal-bin \
  python3-gdal \
  ibgdal-dev \
  libproj-dev
....

And I think the lib's name is libgdal-dev instead of ibgdal-dev

Answer 3

I would second Val's answer to use the 'ubuntugis' PPA and have more recent GDAL etc. libraries available. At least that did the trick for me. I should add that I went off the base GAE flex image (which is based on Ubuntu 16.04 LTS xenial) so that GAE's health checks work as intended. They can be hard to debug too.

You also need to use GAE's custom/flex environment in your app.yaml (assuming your project is called "my-app" so check where wsgi.py is located):

runtime: custom
env: flex
entrypoint: gunicorn -b :$PORT my-app.wsgi

runtime_config:
    python_version: 3

Give this Dockerfile a try:

FROM gcr.io/google-appengine/python
ENV PYTHONUNBUFFERED 1
ENV DEBIAN_FRONTEND noninteractive

RUN apt -y update && apt -y upgrade\
    && apt-get install -y software-properties-common \
    && add-apt-repository -y ppa:ubuntugis/ppa \
    && apt -y update && apt -y upgrade\
    && apt-get -y install \
    gdal-bin \
    libgdal-dev \
    python3-gdal  \ 
    && apt-get autoremove -y \
    && apt-get autoclean -y \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

RUN virtualenv /env -p python3.7
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

ADD requirements.txt /app/requirements.txt
RUN python3 -m pip install -r /app/requirements.txt 
ADD . /app/
WORKDIR /app
CMD gunicorn -b :$PORT my-app.wsgi

Note that you mention in one of your comments above to add the requirements.txt to your container but you actually don't do that in the Dockerfile.

Python 3.7 comes with the GAE base image, but if you want to go for 3.8 see the Dockerfile describe here