在Amazon ec2 ubuntu实例上使用enthought Python安装lxml

Question

I successfully installed enthought Python on ubuntu (running in Amazon EC2 instance). However, I get the following error (log inserted) when trying to install the lxml package

/usr/local/bin/pip run on Sat Mar 30 22:05:14 2013
Downloading/unpacking lxml

Running setup.py egg_info for package lxml

/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'

 warnings.warn(msg)

 Building lxml version 3.1.1.

 Building without Cython.

 Using build configuration of libxslt 1.1.26

 Building against libxml2/libxslt in the following directory: /usr/lib/x86_64-linux-gnu

 running egg_info
 writing pip-egg-info/lxml.egg-info/PKG-INFO
 writing top-level names to pip-egg-info/lxml.egg-info/top_level.txt
 writing dependency_links to pip-egg-info/lxml.egg-info/dependency_links.txt
 warning: manifest_maker: standard file '-c' not found


 reading manifest file 'pip-egg-info/lxml.egg-info/SOURCES.txt'
 reading manifest template 'MANIFEST.in'
 warning: no files found matching '*.txt' under directory 'src/lxml/tests'

 writing manifest file 'pip-egg-info/lxml.egg-info/SOURCES.txt'
 Source in /tmp/pip-build-root/lxml has version 3.1.1, which satisfies requirement lxml
 Installing collected packages: lxml

 Running setup.py install for lxml

Running command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-1bVipz-record/install-record.txt --single-version-externally-managed
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'

warnings.warn(msg)

Building lxml version 3.1.1.

Building without Cython.

Using build configuration of libxslt 1.1.26

Building against libxml2/libxslt in the following directory: /usr/lib/x86_64-linux-gnu

running install
running build
running build_py
copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.7/lxml/includes
running build_ext
building 'lxml.etree' extension

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o

src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__getFilenameForFile’:

src/lxml/lxml.etree.c:26852:7: warning: variable ‘__pyx_clineno’ set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c:26851:15: warning: variable ‘__pyx_filename’ set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c:26850:7: warning: variable ‘__pyx_lineno’ set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c: In function ‘__pyx_pf_4lxml_5etree_4XSLT_18__call__’:

src/lxml/lxml.etree.c:138240:81: warning: passing argument 1 of ‘__pyx_f_4lxml_5etree_12_XSLTContext__copy’ from incompatible pointer type [enabled by default]

src/lxml/lxml.etree.c:136196:52: note: expected ‘struct __pyx_obj_4lxml_5etree__XSLTContext *’ but argument is of type ‘struct __pyx_obj_4lxml_5etree__BaseContext *’

src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__copyXSLT’:

src/lxml/lxml.etree.c:139634:79: warning: passing argument 1 of ‘__pyx_f_4lxml_5etree_12_XSLTContext__copy’ from incompatible pointer type [enabled by default]

src/lxml/lxml.etree.c:136196:52: note: expected ‘struct __pyx_obj_4lxml_5etree__XSLTContext *’ but argument is of type ‘struct __pyx_obj_4lxml_5etree__BaseContext *’

src/lxml/lxml.etree.c: At top level:

src/lxml/lxml.etree.c:12385:13: warning: ‘__pyx_f_4lxml_5etree_displayNode’ defined but not used [-Wunused-function]

gcc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

error: command 'gcc' failed with exit status 4

Complete output from command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-1bVipz-record/install-record.txt --single-version-externally-managed:

/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'

warnings.warn(msg)

Building lxml version 3.1.1.

Building without Cython.

Using build configuration of libxslt 1.1.26

Building against libxml2/libxslt in the following directory: /usr/lib/x86_64-linux-gnu

running install

running build

running build_py

copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.7/lxml/includes

running build_ext

building 'lxml.etree' extension

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o

src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__getFilenameForFile’:

src/lxml/lxml.etree.c:26852:7: warning: variable ‘__pyx_clineno’ set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c:26851:15: warning: variable ‘__pyx_filename’ set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c:26850:7: warning: variable ‘__pyx_lineno’ set but not used [-Wunused-but-set-variable]

src/lxml/lxml.etree.c: In function ‘__pyx_pf_4lxml_5etree_4XSLT_18__call__’:

src/lxml/lxml.etree.c:138240:81: warning: passing argument 1 of ‘__pyx_f_4lxml_5etree_12_XSLTContext__copy’ from incompatible pointer type [enabled by default]

src/lxml/lxml.etree.c:136196:52: note: expected ‘struct __pyx_obj_4lxml_5etree__XSLTContext *’ but argument is of type ‘struct __pyx_obj_4lxml_5etree__BaseContext *’

src/lxml/lxml.etree.c: In function ‘__pyx_f_4lxml_5etree__copyXSLT’:

src/lxml/lxml.etree.c:139634:79: warning: passing argument 1 of ‘__pyx_f_4lxml_5etree_12_XSLTContext__copy’ from incompatible pointer type [enabled by default]

src/lxml/lxml.etree.c:136196:52: note: expected ‘struct __pyx_obj_4lxml_5etree__XSLTContext *’ but argument is of type ‘struct __pyx_obj_4lxml_5etree__BaseContext *’

src/lxml/lxml.etree.c: At top level:

src/lxml/lxml.etree.c:12385:13: warning: ‘__pyx_f_4lxml_5etree_displayNode’ defined but not used [-Wunused-function]

gcc: internal compiler error: Killed (program cc1)

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

error: command 'gcc' failed with exit status 4

---

Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-1bVipz-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/lxml

Exception information:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 271, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 1185, in install
requirement.install(install_options, global_options, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pip/req.py", line 592, in install
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
File "/usr/local/lib/python2.7/dist-packages/pip/util.py", line 662, in call_subprocess
% (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-root/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-1bVipz-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip-build-root/lxml

Answer 1

I found the same issue on my instance. It turned out to be that compiling lxml takes a lot of memory and my instance was running out of RAM, so the kernel was killing the compiler.

To check if you have the same issue, try running dmesg|tail after the failure. You'll see some lines like:

[2242482.581361] Out of memory: Kill process 8306 (cc1) score 432 or sacrifice child
[2242482.581375] Killed process 8306 (cc1) total-vm:298572kB, anon-rss:260756kB, file-rss:4kB

Answer 2

To install lxml in an AWS micro-instance try to follow these steps:

sudo yum install python-devel
sudo yum install libxml2-devel
sudo yum install libxslt-devel  
sudo yum install gcc        
sudo easy_install pip
sudo pip install lxml

Remember that Amazon keeps just some selected files available to install, sometimes with different names. If you look for:

yum info python-dev

you will find just an error:

Error: No matching Packages to list

but...

yum info python-devel

will show what you need

Name        : python-devel
Arch        : noarch
Epoch       : 1
Version     : 2.6
Release     : 2.26.amzn1
Size        : 0.0  
Repo        : installed
From repo   : amzn-main
Summary     : The libraries and header files needed for Python
            : development.
URL         : http://www.python.org/
License     : PSF
Description : This is a macro package that pulls in via
            : dependencies the system default python development
            : extensions.

Answer 3

To compile lxml you need to have a couple of development libraries installed first. On Ubuntu you can install these with:

sudo apt-get install libxml2-dev libxslt-dev

Answer 4

To add on Daniel's answer, besides installing libxslt-devel, python-devel, libxml2-devel

sudo yum groupinstall -y development
sudo yum install libxslt-devel, python-devel, libxml2-devel

if you got message like this:

[2242482.581361] Out of memory: Kill process 8306 (cc1) score 432 or sacrifice child 
[2242482.581375] Killed process 8306 (cc1) total-vm:298572kB, anon-rss:260756kB, file-rss:4kB

you can create a swap file to top up your memory, for example to add in 512Mb of swap file:

sudo dd if=/dev/zero of=/swapfile bs=1024 count=512k
sudo mkswap /swapfile
sudo swapon /swapfile
compile lxml

source: encountered this problem and finally fixed it.