Installing `dulwich` gives `error: command 'clang' failed with exit status 1`

Question

When I try to execute pip install dulwich , I end up with a wall of red complaining about clang . Here's the last bit:

    Please check your Xcode installation
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/openssl/include -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/alaird/Developer/py3venv/include -I/usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c dulwich/_objects.c -o build/temp.macosx-10.14-x86_64-3.7/dulwich/_objects.o
    clang: warning: no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
    In file included from dulwich/_objects.c:20:
    /usr/local/Cellar/python/3.7.4_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/Python.h:25:10: fatal error: 'stdio.h' file not found
    #include <stdio.h>
             ^~~~~~~~~
    1 error generated.
    error: command 'clang' failed with exit status 1
    ----------------------------------------
ERROR: Command "/Users/alaird/Developer/py3venv/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/private/var/folders/63/r_rf7mhj14b4c42mfkkl06t8rg1b1r/T/pip-install-b9wlh7xh/dulwich/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/63/r_rf7mhj14b4c42mfkkl06t8rg1b1r/T/pip-record-_v0dq_sp/install-record.txt --single-version-externally-managed --compile --install-headers /Users/alaird/Developer/py3venv/include/site/python3.7/dulwich" failed with error code 1 in /private/var/folders/63/r_rf7mhj14b4c42mfkkl06t8rg1b1r/T/pip-install-b9wlh7xh/dulwich/

I've gone down a lot of " clang failed with exit status 1" rabbit holes on Google, problem is that's too generic. To answer a few questions:

1. Yes, Xcode is the latest version, and command-line tools have been installed (I'm on OS X 10.14.6, so this is now managed through app updates).
2. This was a previously working installation. This error began occuring after I did a make clean , which blew away my venv .
3. Attempting to install the latest setuptools runs in to the same error, so feels like the error pointing to Xcode/my build environment (and not dulwich ) is the correct culprit.
4. Doing a brew update also gives me the same clang barf. I think something got hosed when I upgraded to Mojave.

Ideas?

Answer 1

Ah-ha!

I started looking at the warning instead of the error:

no such sysroot directory: '/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]

And this led me to notice a SDK version number was hardcoded... which seems stupid. So I built a symlink:

ln -s /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

Depending on your system configuration, you may need to build a different symlink (there was also a MacOSX10.15.sdk folder).

Then it worked! No issues with pip (including installing dulwich and pycrypto ), no issues with brew , and all is well with my development environment again!

I also did:

export CFLAGS="-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk"

However, I don't believe that was what did it, since after starting a new session all still worked. So it was probably the symlink above that resolved it. Hope this helps others!