在Ubuntu 16.04 Gdal上安装CartoDB编译错误

Question

I followed the installation steps in https://gist.github.com/azvoleff/f8f06d22a8a4d89401e09d6607a5ecc4

to install cartoDB to Ubuntu 16.04.

everything is fine and installed correctly (although i used RVM instead of getting ruby's source code and compile it myself as suggested from the steps).

but when running: pip install --no-use-wheel -r python_requirements.txt

there is an error, says about gdal, but it's different with the error mentioned in the steps.

DEPRECATION: --no-use-wheel is deprecated and will be removed in the future.  Please use --no-binary :all: instead.
Requirement already satisfied: chardet==2.3.0 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 1))
Requirement already satisfied: argparse==1.2.1 in /usr/lib/python2.7 (from -r python_requirements.txt (line 2))
Requirement already satisfied: brewery==0.6 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 3))
Requirement already satisfied: redis==2.4.9 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 4))
Requirement already satisfied: python-varnish==0.1.2 in /usr/local/lib/python2.7/dist-packages (from -r python_requirements.txt (line 5))
Collecting gdal==1.10.0 (from -r python_requirements.txt (line 6))
  Using cached GDAL-1.10.0.tar.gz
Collecting csvkit==0.9.0 (from -r python_requirements.txt (line 7))
  Using cached csvkit-0.9.0.tar.gz
Collecting openpyxl==2.1.3 (from -r python_requirements.txt (line 8))
  Using cached openpyxl-2.1.3.tar.gz
Collecting xlrd>=0.7.1 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Using cached xlrd-1.1.0.tar.gz
Collecting sqlalchemy>=0.6.6 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Using cached SQLAlchemy-1.2.0.tar.gz
Collecting six>=1.6.1 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Using cached six-1.11.0.tar.gz
Collecting python-dateutil==2.2 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Using cached python-dateutil-2.2.tar.gz
Collecting dbf==0.94.003 (from csvkit==0.9.0->-r python_requirements.txt (line 7))
  Using cached dbf-0.94.003.tar.gz
Collecting jdcal (from openpyxl==2.1.3->-r python_requirements.txt (line 8))
  Using cached jdcal-1.3.tar.gz
Skipping bdist_wheel for gdal, due to binaries being disabled for it.
Skipping bdist_wheel for csvkit, due to binaries being disabled for it.
Skipping bdist_wheel for openpyxl, due to binaries being disabled for it.
Skipping bdist_wheel for xlrd, due to binaries being disabled for it.
Skipping bdist_wheel for sqlalchemy, due to binaries being disabled for it.
Skipping bdist_wheel for six, due to binaries being disabled for it.
Skipping bdist_wheel for python-dateutil, due to binaries being disabled for it.
Skipping bdist_wheel for dbf, due to binaries being disabled for it.
Skipping bdist_wheel for jdcal, due to binaries being disabled for it.
Installing collected packages: gdal, xlrd, sqlalchemy, jdcal, openpyxl, six, python-dateutil, dbf, csvkit
  Found existing installation: GDAL 2.2.0
    Uninstalling GDAL-2.2.0:
      Successfully uninstalled GDAL-2.2.0
  Running setup.py install for gdal ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-U2YlBY/gdal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-UB4Lg0-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    copying gdal.py -> build/lib.linux-x86_64-2.7
    copying ogr.py -> build/lib.linux-x86_64-2.7
    copying osr.py -> build/lib.linux-x86_64-2.7
    copying gdalconst.py -> build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdalconst.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdal_array.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdalnumeric.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/osr.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/__init__.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/ogr.py -> build/lib.linux-x86_64-2.7/osgeo
    copying osgeo/gdal.py -> build/lib.linux-x86_64-2.7/osgeo
    running build_ext
    building 'osgeo._gdal' extension
    creating build/temp.linux-x86_64-2.7
    creating build/temp.linux-x86_64-2.7/extensions
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I../../port -I../../gcore -I../../alg -I../../ogr/ -I/usr/include/python2.7 -I. -I/usr/include -c extensions/gdal_wrap.cpp -o build/temp.linux-x86_64-2.7/extensions/gdal_wrap.o
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
    extensions/gdal_wrap.cpp:2853:22: fatal error: cpl_port.h: No such file or directory
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
  Rolling back uninstall of GDAL
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-U2YlBY/gdal/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-UB4Lg0-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-U2YlBY/gdal/

i'm new in ubuntu, previously i tried to set this up on centos (because i'm more familiar with centos), but it was worse than using ubuntu because many dependencies are missing / out of date.

do anyone have any suggestion on how to successfully setup cartoDB in ubuntu / any other linux based machine? (or what should i do with this compile error?)

i guess i should get gdal version 1.10.0 ? but how to do that in ubuntu...

[UPDATE-1]

Maybe it's because cpl_port.h not found from this error extensions/gdal_wrap.cpp:2853:22: fatal error: cpl_port.h: No such file or directory

so, i did install libgdal-dev per Oz123 suggests.

apt-get install libgdal-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
libgdal-dev is already the newest version (2.2.2+dfsg-1~xenial1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

and my GDAL version is

gdalinfo --version
GDAL 2.2.2, released 2017/09/15

and sadly, error is still the same. i'm still trying to find out how to compile / change installed GDAL version to 1.10.0 because i assume the requirement is GDAL 1.10.0 and what i have is 2.2.2, that's why it tries to compile again, am i correct with this?

...
Collecting gdal==1.10.0 (from -r python_requirements.txt (line 6))
Using cached GDAL-1.10.0.tar.gz
...
Installing collected packages: gdal, xlrd, sqlalchemy, jdcal, openpyxl, six, python-dateutil, dbf, csvkit
Found existing installation: GDAL 2.2.0
Uninstalling GDAL-2.2.0:
  Successfully uninstalled GDAL-2.2.0
Running setup.py install for gdal ... error
...

[UPDATE-2-ANSWER FOR THIS SPECIFIC PROBLEM]

thx to Oz123, I execute:

export CPLUS_INCLUDE_PATH=/usr/include/gdal
export C_INCLUDE_PATH=/usr/include/gdal

and retry the compile, and it fix this cpl_port.h problem. but the installation still failed at the same installation step (GDAL compile), but i think i should ask this in different thread to keep this stack clean.

Answer 1

This is probably because you are missing libgdal-dev ( found through ubuntu package content search ).

When ever you install a package with hard coded C dependecies you should make sure you have the build-dependencies.

I guess I should get gdal version 1.10.0 ? but how to do that in ubuntu...

If you followed this step:

sudo add-apt-repository ppa:ubuntugis/ppa
sudo apt-get update

You will get a later version of GDAL.

Never the less, you should install the build dependency, libgdal-dev with sudo apt-get install libgdal-dev , the repeat pip install --no-use-wheel -r python_requirements.txt , if you get more errors, look for the missing header file in the ubuntu packages content search pages, install the package and repeat until complete.

You should also pass the paths to pip:

export CPLUS_INCLUDE_PATH=/usr/include/gdal

export C_INCLUDE_PATH=/usr/include/gdal

pip install --no-use-wheel -r python_requirements.txt