Compile hadoop native libraries in Mac OS Catalina. error: unknown type name 'constexpr'. alias declarations are a C++11 extension

Question

I'm trying to compile Hadoop native libraries in Unix, these is what I'm using:

1. Hadoop: branch-3.2 ( here is the Hadoop Repository )
2. macOS: Catalina Version 10.15.5
3. Protobuf: libprotoc 2.5.0
4. OpenSSL 1.0.2t 10 Sep 2019
5. gcc: Apple clang version 11.0.3 (clang-1103.0.32.62) Target: x86_64-apple-darwin19.5.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin
6. java version "1.8.0_251" (build 1.8.0_251-b08)

I followed all steps here: Compiling Hadoop on MAC After fixing a couple of problems with openssl and protobuf now it looks like the compilation of Apache Hadoop MapReduce NativeTask is failing due to a dependency with C++11 .

Do I need to specify somehow to maven the CXXFLAGS parameter set to -std=c++11? How do I do it? Or am I missing some configuration in my gcc?

I tried setting an alias to gcc so then it immediately is called with the flags like it's explained here: How to make clang support C++ 11 by default , but still doesn't work.

So far these are one of the many similar errors I have:

[WARNING] /usr/local/include/snappy-stubs-public.h:61:16: warning: alias declarations are a C++11 extension [-Wc++11-extensions]

[WARNING] using uint64 = std::uint64_t;

[WARNING] In file included from /Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/test/TestCompressions.cc:29:

[WARNING] /usr/local/include/snappy.h:197:10: error: unknown type name 'constexpr'

[WARNING]   static constexpr int kBlockLog = 16;

[INFO] Apache Hadoop MapReduce NativeTask ................. FAILURE [  1.995 s]

And after reading the output more carefully, I think this is the command that ends up throwing the error:

[WARNING] /Library/Developer/CommandLineTools/usr/bin/make  -f CMakeFiles/nttest.dir/build.make CMakeFiles/nttest.dir/build
[WARNING] [ 75%] Building CXX object CMakeFiles/nttest.dir/main/native/test/TestCompressions.cc.o
[WARNING] /Library/Developer/CommandLineTools/usr/bin/c++   -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/javah -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/util -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/lib -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/test -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src -I/Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native -I/Library/Java/JavaVirtualMachines/jdk1.8.0_251.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/jdk1.8.0_251.jdk/Contents/Home/include/darwin -I/usr/local/include -isystem /Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/../../../../hadoop-common-project/hadoop-common/src/main/native/gtest/include  -g -O2 -Wall -pthread -D_FILE_OFFSET_BITS=64 -DNDEBUG -DSIMPLE_MEMCPY -fno-strict-aliasing -fsigned-char -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk   -o CMakeFiles/nttest.dir/main/native/test/TestCompressions.cc.o -c /Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/test/TestCompressions.cc
[WARNING] In file included from /Users/josh/Dev/hadoop/repo/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/test/TestCompressions.cc:29:
[WARNING] In file included from /usr/local/include/snappy.h:45:
[WARNING] /usr/local/include/snappy-stubs-public.h:54:14: warning: alias declarations are a C++11 extension [-Wc++11-extensions]
[WARNING] using int8 = std::int8_t;

THANK YOU!

Answer 1

I also had difficulty building the hadoop-native libs on OSX with clang . I had more success switching to using gcc 10.

First install with homebrew:

brew install gcc

This should put the gcc binaries in the path, but they'll be gcc-10 , g++-10 etc. So setting some environment magic should casue CMake to use these rather than the built-in clang binaries:

export CC=$(which gcc-10)
export CXX=$(which g++-10)
export CPP=$(which cpp-10)
export LD=$(which gcc-10)