Caffe install error on Ubuntu 16.04 for CUDA 8.0 & cuDNN 5.1

Question

I am using a GTX 1080 backed UBUNUTU 16.04 Xenial PC for deep learning. However, I am facing little problem on compiling caffe from BLVC or NVIDIA source. After installing all dependencies and linking global variables, Im still missing something to compile caffe. I've built OpenCV 3.1.0 & OpenBLAS etc. Now cloning from https://github.com/BVLC/caffe and entering

cd caffe
 mkdir build
cd build 
cmake ..

gives me the following error --

-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   system
--   thread
--   filesystem
-- Looking for include file pthread.h
-- Looking for include file pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found GFlags: /usr/include  
-- Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Found Glog: /usr/include  
-- Found glog    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Found PROTOBUF: /usr/lib/x86_64-linux-gnu/libprotobuf.so  
-- Found PROTOBUF Compiler: /usr/bin/protoc
CMake Error at /usr/local/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:138 (message):
  Could NOT find HDF5 (missing: HDF5_INCLUDE_DIRS)
Call Stack (most recent call first):
  /usr/local/share/cmake-3.2/Modules/FindPackageHandleStandardArgs.cmake:374 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/share/cmake-3.2/Modules/FindHDF5.cmake:360 (find_package_handle_standard_args)
  cmake/Dependencies.cmake:27 (find_package)
  CMakeLists.txt:43 (include)


-- Configuring incomplete, errors occurred!
See also "/home/xhuv/testcaffe/caffe/build/CMakeFiles/CMakeOutput.log".
See also "/home/xhuv/testcaffe/caffe/build/CMakeFiles/CMakeError.log".

I'm using Python 3.5.2. :: Anaconda 4.2.0 (64-bit), Python version 2.7 also installed. I have edited the Makefile.config according to the following

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
 USE_OPENCV := 3
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#   You should not set this flag if you will be reading LMDBs with any
#   possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
        -gencode arch=compute_20,code=sm_21 \
        -gencode arch=compute_30,code=sm_30 \
        -gencode arch=compute_35,code=sm_35 \
        -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/local/include
BLAS_LIB := /usr/local/lib

# Homebrew puts openblas in a directory that is not on the standard search path
#BLAS_INCLUDE := $(shell brew --prefix openblas)/include
#BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
         $(ANACONDA_HOME)/include/python2.7 \
         $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
#WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
LIBRARY_DIRS += /usr/lib/x86_64-linux-gnu/
LIBRARY_DIRS += $(ANACONDA_HOME)/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
#INCLUDE_DIRS += $(shell brew --prefix)/include
#LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @ 

# shared object suffix name to differentiate branches
LIBRARY_NAME_SUFFIX := -nv

What could be the possible error? Please help!!

Answer 1

您首先需要使用此命令安装HD5F sudo apt-get install libhdf5-serial-dev并将添加的库目录添加到此+LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/但库将在其扩展名中添加其版本，因此您需要创建具有官方名称的符号链接，caffe在执行类似$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.10 /usr/lib/x86_64-linux-gnu/libhdf5.so $ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.10 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so构建时将查找$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.10 /usr/lib/x86_64-linux-gnu/libhdf5.so $ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.10 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so 。希望对您$ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial.so.10 /usr/lib/x86_64-linux-gnu/libhdf5.so $ sudo ln -s /usr/lib/x86_64-linux-gnu/libhdf5_serial_hl.so.10 /usr/lib/x86_64-linux-gnu/libhdf5_hl.so帮助

Answer 2

Caffe installation can be challenging specific to the architecture. The problem with anaconda is that it does not use the identical protobuf package by the python for google protocol support which is required by the Caffe. For Anaconda_Caffe installation refer to this repository .

Answer 3

The install directories for HDF5 on Ubuntu 16.04 don't seem to play nice with the cmake FindHDF5 module. I spent a good several hours digging around for a way to fix this "nicely", but ended up just patching the cmake/Dependencies.cmake file to get caffe to compile properly.

In any case, here's the patch: ubuntu1604_caffe_hdf5.patch

And if you're looking for a quick way to include it into a build script (like a Dockerfile), I've made a git.io link you can run in your base caffe directory to patch cmake/Dependencies.cmake :

wget -O- https://git.io/vHcP3 | patch -p0