Cannot install lxml using pip

Question

I have tried to install lxml python library on Ubuntu 14.04 using pip install lxml .

It have failed and the output was (brought only the end here):

...
...
...

src/lxml/lxml.etree.c:8:22: fatal error: pyconfig.h: No such file or directory

 #include "pyconfig.h"

                      ^

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

I have searched the problem and found that I should install some stuff using:

apt-get install libxml2-dev libxslt-dev libxslt1-dev python-dev

Then I have retried to install using pip install lxml . The installation failed again, now with a different error. Trying pip --default-timeout=300 install lxml changed nothing.

The new error is:

...
...
...

creating build/temp.linux-x86_64-2.7/src/lxml

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip_build_root/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w

x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so

/usr/bin/ld: cannot find -lz

collect2: error: ld returned 1 exit status

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-PVygtI-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml
Storing debug log for failure in /root/.pip/pip.log

Additional information from /root/.pip/pip.log file:

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1436, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pip/req.py", line 707, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/lib/python2.7/dist-packages/pip/util.py", line 715, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip_build_root/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-Jj2y9w-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/lxml

However, I can install lxml by apt-get install python-lxml , and also install locally after downloading manually from lxml's website but I want to install using pip.

Answer 1

This error...

/usr/bin/ld: cannot find -lz

...suggests that you also need to install zlib1g-dev :

apt-get install zlib1g-dev

The lxml module has a fairly large number of dependencies. It's often much easier to use the one provided by the system package manager, unless you need a feature that is only available if you build from source.

In general, for this sort of error, you need to find a package that provides the missing shared library. This will be lib<SOMETHING>.so , where <SOMETHING> is whatever follows the -l in the error message.

You can use the apt-file command to help locate the appropriate package. For example:

# apt-file search libz.so
lib32z1: /usr/lib32/libz.so.1
lib32z1: /usr/lib32/libz.so.1.2.8
lib32z1-dev: /usr/lib32/libz.so
libx32z1: /usr/libx32/libz.so.1
libx32z1: /usr/libx32/libz.so.1.2.8
libx32z1-dev: /usr/libx32/libz.so
zlib1g: /lib/x86_64-linux-gnu/libz.so.1
zlib1g: /lib/x86_64-linux-gnu/libz.so.1.2.8
zlib1g-dev: /usr/lib/x86_64-linux-gnu/libz.so

The last entry in the list there is for .../libz.so , provided by the zlib1g-dev package.