Python build using wrong version of GCC on OS X

Question

I am attempting to build the python package pycrypto. OS X has gcc-4.2 installed and not gcc-4.0, but python continues to attempt to use gcc-4.0. How can I get it to use gcc-4.2? Or should I go about this a different way.

I am getting the following error:

bash-3.2$ 
bash-3.2$ sudo python setup.py build
running build
running build_py
running build_ext
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash.MD2' extension
gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -arch ppc -arch i386 -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/MD2.c -o build/temp.macosx-10.3-fat-2.6/src/MD2.o
unable to execute gcc-4.0: No such file or directory
error: command 'gcc-4.0' failed with exit status 1
bash-3.2$ 
bash-3.2$ 
bash-3.2$ 

I am using Mac OS X 10.6.7 with python 2.6.6 and XCode is installed.

EDIT: If I add CC=gcc-4.2, then I still get the error:

bash-3.2$ 
bash-3.2$ export CC=gcc-4.2
bash-3.2$ python setup.py build
running build
running build_py
running build_ext
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash.MD2' extension
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -std=c99 -O3 -fomit-frame-pointer -arch i386 -arch x86_64 -Isrc/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/MD2.c -o build/temp.macosx-10.3-intel-2.6/src/MD2.o
gcc-4.0 -g -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 build/temp.macosx-10.3-intel-2.6/src/MD2.o -o build/lib.macosx-10.3-intel-2.6/Crypto/Hash/MD2.so
unable to execute gcc-4.0: No such file or directory
error: command 'gcc-4.0' failed with exit status 1
bash-3.2$ 

EDIT: It seems that using sudo makes a difference here.

I tried using both CC and CXX as suggest by Adam, and I get the following error without sudo:

bash-3.2$ python setup.py build
running build
running build_py
creating build/lib.macosx-10.3-fat-2.6
creating build/lib.macosx-10.3-fat-2.6/Crypto
copying lib/Crypto/__init__.py -> build/lib.macosx-10.3-fat-2.6/Crypto
copying lib/Crypto/pct_warnings.py -> build/lib.macosx-10.3-fat-2.6/Crypto
...
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash.MD2' extension
creating build/temp.macosx-10.3-fat-2.6
creating build/temp.macosx-10.3-fat-2.6/src
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -arch ppc -arch i386 -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/MD2.c -o build/temp.macosx-10.3-fat-2.6/src/MD2.o
/usr/libexec/gcc/powerpc-apple-darwin10/4.2.1/as: assembler (/usr/bin/../libexec/gcc/darwin/ppc/as or /usr/bin/../local/libexec/gcc/darwin/ppc/as) for architecture ppc not installed
Installed assemblers are:
/usr/bin/../libexec/gcc/darwin/x86_64/as for architecture x86_64
/usr/bin/../libexec/gcc/darwin/i386/as for architecture i386
lipo: can't open input file: /var/tmp//ccxan625.out (No such file or directory)
error: command 'gcc-4.2' failed with exit status 1

If I use sudo, I get the following error where it tries to use 4.0:

bash-3.2$ sudo python setup.py build
Password:
running build
running build_py
running build_ext
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash.MD2' extension
gcc-4.0 -fno-strict-aliasing -fno-common -dynamic -arch ppc -arch i386 -std=c99 -O3 -fomit-frame-pointer -Isrc/ -I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/MD2.c -o build/temp.macosx-10.3-fat-2.6/src/MD2.o
unable to execute gcc-4.0: No such file or directory
error: command 'gcc-4.0' failed with exit status 1
bash-3.2$ 
bash-3.2$ 

Does this extra info make it more obvious what is going on and how to fix it? Any idea why the call without sudo is getting that other error?

Answer 1

Based on the path shown ( /Library/Frameworks/Python.framework/Versions/2.6 ), it appears you have installed a 32-bit-only Python 2.6, perhaps using a python.org installer. When you build a Python package that includes a C extension module, the Python Distutils included with that Python instance will attempt to use the same version of gcc and the same CPU architectures that that Python was built with. Apparently you have installed the new cutting-edge Xcode 4 which no longer includes gcc-4.0 or ppc support. The Python versions you are using was built with the Xcode 3 tools included with Mac OS X 10.6. You may be able to work around it by overriding the compiler choice with:

export CC=gcc-4.2
python setup.py build
sudo python setup.py install

EDIT:

It looks like that is not going to work for pycrypto ; its build is too compex. If you don't mind using the Apple-supplied Python 2.6 in OS X 10.6, this should work:

export ARCHFLAGS='-arch i386 -arch x86_64'
/usr/bin/python2.6 setup.py build

Another option would be to install the 64-bit/32-bit Python 2.7 installer from python.org . That is built with gcc-4.2 and is Intel-only so there shouldn't be any problems when using it with Xcode 4.

UPDATE:

Here are the exact steps I used with Xcode 3. They should work as well with Xcode 4 installed:

$ mkdir p
$ cd p
$ curl -O http://ftp.dlitz.net/pub/dlitz/crypto/pycrypto/pycrypto-2.3.tar.gz
$ tar xf pycrypto-2.3.tar.gz 
$ cd pycrypto-2.3/
$ export ARCHFLAGS='-arch i386 -arch x86_64'
$ /usr/bin/python2.6 setup.py build
running build
running build_py
creating build
creating build/lib.macosx-10.6-universal-2.6
creating build/lib.macosx-10.6-universal-2.6/Crypto
copying lib/Crypto/__init__.py -> build/lib.macosx-10.6-universal-2.6/Crypto
[...]
running build_ext
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
building 'Crypto.Hash.MD2' extension
creating build/temp.macosx-10.6-universal-2.6
creating build/temp.macosx-10.6-universal-2.6/src
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -fwrapv -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe -std=c99 -O3 -fomit-frame-pointer -arch i386 -arch x86_64 -Isrc/ -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/MD2.c -o build/temp.macosx-10.6-universal-2.6/src/MD2.o
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 build/temp.macosx-10.6-universal-2.6/src/MD2.o -o build/lib.macosx-10.6-universal-2.6/Crypto/Hash/MD2.so
[...]
building 'Crypto.Util._counter' extension
gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -fwrapv -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe -std=c99 -O3 -fomit-frame-pointer -arch i386 -arch x86_64 -Isrc/ -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c src/_counter.c -o build/temp.macosx-10.6-universal-2.6/src/_counter.o
gcc-4.2 -Wl,-F. -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 build/temp.macosx-10.6-universal-2.6/src/_counter.o -o build/lib.macosx-10.6-universal-2.6/Crypto/Util/_counter.so
$ /usr/bin/python2.6 setup.py install
running install
running build
running build_py
running build_ext
warning: GMP library not found; Not building Crypto.PublicKey._fastmath.
running install_lib
creating /Library/Python/2.6/site-packages/Crypto
[...]
byte-compiling /Library/Python/2.6/site-packages/Crypto/pct_warnings.py to pct_warnings.pyc
running install_egg_info
Writing /Library/Python/2.6/site-packages/pycrypto-2.3-py2.6.egg-info
$ /usr/bin/python2.6 setup.py test
running test
............................................................................................[...]
----------------------------------------------------------------------
Ran 902 tests in 42.612s

OK
$ cd 
$ /usr/bin/python2.6
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) 
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from Crypto.Hash import MD5
>>> m = MD5.new()
>>> m.update('abc')
>>> m.digest()
'\x90\x01P\x98<\xd2O\xb0\xd6\x96?}(\xe1\x7fr'
>>> m.hexdigest()
'900150983cd24fb0d6963f7d28e17f72'
>>> ^D

Answer 2

BTW, this is a distutils problem, Python itself doesn't compile anything.

As you've discovered, you can override the compiler with the CC environment variable. You can override the linker used with the CXX environment variable. I would also like to know why distutils acts this way, but so it does.

Answer 3

Focus on this error:

unable to execute gcc-4.0: No such file or directory
error: command 'gcc-4.0' failed with exit status 1

The compiler is telling you EXACTLY what the problem is. gcc-4.0 (and gcc-4.2 ) is not in your PATH . HINT: which gcc-4.2 , eg:

% which gcc-4.2
/usr/bin/gcc-4.2

Assuming yours is in the same location, I'm not sure why /usr/bin isn't in your PATH , but there you have it!

Find out what your PATH actually is:

% echo $PATH
/opt/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/bin/X11:/sw/bin:/sw/sbin

Fix your path and you'll be on your path to enlightenment.