Fail to install lxml on ubuntu

Question

I am attempting to install lxml inside a virtualenv on ubuntu 14.04.

$ uname -a
Linux foobar 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.1 LTS
Release:    14.04
Codename:   trusty

I have the dependencies installed, as specified in the lxml documentation. Here are the specific packages I have installed:

$ dpkg-query -l | grep libxml
ii  libxml-commons-external-java              1.4.01-2build1                             all          XML Commons external code - DOM, SAX, and JAXP, etc
ii  libxml-commons-resolver1.1-java           1.2-7build1                                all          XML entity and URI resolver library
ii  libxml-parser-perl                        2.41-1build3                               amd64        Perl module for parsing XML files
ii  libxml2:amd64                             2.9.1+dfsg1-3ubuntu4.4                     amd64        GNOME XML library
ii  libxml2-dev:amd64                         2.9.1+dfsg1-3ubuntu4.4                     amd64        Development files for the GNOME XML library
ii  python-lxml                               3.3.3-1ubuntu0.1                           amd64        pythonic binding for the libxml2 and libxslt libraries
$ dpkg-query -l | grep libxslt
ii  libxslt1-dev:amd64                        1.1.28-2build1                             amd64        XSLT 1.0 processing library - development kit
ii  libxslt1.1:amd64                          1.1.28-2build1                             amd64        XSLT 1.0 processing library - runtime library
ii  python-lxml                               3.3.3-1ubuntu0.1                           amd64        pythonic binding for the libxml2 and libxslt libraries
$ dpkg-query -l | grep python-dev
ii  libpython-dev:amd64                       2.7.5-5ubuntu3                             amd64        header files and a static library for Python (default)
ii  python-dev                                2.7.5-5ubuntu3                             amd64        header files and a static library for Python (default)

When I attempt to install inside a virtualenv:

$ virtualenv venv
New python executable in venv/bin/python
Installing setuptools, pip, wheel...done.
~$ source venv/bin/activate
(venv)~$ export PATH=$PATH:/usr/lib/gcc/x86_64-linux-gnu/4.8/ # If I don't do this cc1 is not found and the build fails
(venv)~$ pip install lxml
Collecting lxml
  Using cached lxml-3.4.4.tar.gz
Building wheels for collected packages: lxml
  Running setup.py bdist_wheel for lxml
  Complete output from command /home/plasma/venv/bin/python -c "import setuptools;__file__='/tmp/pip-build-Fs3av2/lxml/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /tmp/tmpoWlzvPpip-wheel-:
  /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
    warnings.warn(msg)
  Building lxml version 3.4.4.
  Building without Cython.
  Using build configuration of libxslt 1.1.28
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/__init__.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/doctestcompare.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/builder.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/ElementInclude.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/pyclasslookup.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/_elementpath.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/cssselect.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/sax.py -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/usedoctest.py -> build/lib.linux-x86_64-2.7/lxml
  creating build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/__init__.py -> build/lib.linux-x86_64-2.7/lxml/includes
  creating build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/_diffcommand.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/ElementSoup.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/__init__.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/builder.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/html5parser.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/soupparser.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/_html5builder.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/formfill.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/diff.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/clean.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/_setmixin.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/usedoctest.py -> build/lib.linux-x86_64-2.7/lxml/html
  copying src/lxml/html/defs.py -> build/lib.linux-x86_64-2.7/lxml/html
  creating build/lib.linux-x86_64-2.7/lxml/isoschematron
  copying src/lxml/isoschematron/__init__.py -> build/lib.linux-x86_64-2.7/lxml/isoschematron
  copying src/lxml/lxml.etree.h -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/lxml.etree_api.h -> build/lib.linux-x86_64-2.7/lxml
  copying src/lxml/includes/schematron.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/tree.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/xslt.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/uri.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/relaxng.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/c14n.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/config.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/xpath.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/xinclude.pxd -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.7/lxml/includes
  copying src/lxml/includes/etree_defs.h -> build/lib.linux-x86_64-2.7/lxml/includes
  creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources
  creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/rng
  copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/rng
  creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl
  copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl
  copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl
  creating build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-x86_64-2.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  running build_ext
  building 'lxml.etree' extension
  creating build/temp.linux-x86_64-2.7
  creating build/temp.linux-x86_64-2.7/src
  creating build/temp.linux-x86_64-2.7/src/lxml
  x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-Fs3av2/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
  x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so
  x86_64-linux-gnu-gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

  ----------------------------------------
  Failed building wheel for lxml
Failed to build lxml
Installing collected packages: lxml
  Running setup.py install for lxml
    Complete output from command /home/plasma/venv/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-Fs3av2/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-23Wgk1-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/plasma/venv/include/site/python2.7/lxml:
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    Building lxml version 3.4.4.
    Building without Cython.
    Using build configuration of libxslt 1.1.28
    running install
    running build
    running build_py
    copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-2.7/lxml/includes
    running build_ext
    building 'lxml.etree' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-Fs3av2/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
    x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -lxslt -lexslt -lxml2 -lz -lm -o build/lib.linux-x86_64-2.7/lxml/etree.so
    x86_64-linux-gnu-gcc: fatal error: -fuse-linker-plugin, but liblto_plugin.so not found
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

    ----------------------------------------
Command "/home/plasma/venv/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-Fs3av2/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-23Wgk1-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/plasma/venv/include/site/python2.7/lxml" failed with error code 1 in /tmp/pip-build-Fs3av2/lxml
(venv)~$

However, I do have liblto_plugin.so:

$ find -name liblto_plugin* 2> /dev/null
./usr/lib/gcc/x86_64-linux-gnu/4.8/liblto_plugin.so
./usr/lib/gcc/x86_64-linux-gnu/4.8/liblto_plugin.so.0
./usr/lib/gcc/x86_64-linux-gnu/4.8/liblto_plugin.so.0.0.0

I know I can install lxml using apt-get, but I am trying to have a virtual environment in which I can install these packages so I can have a consistent environment for unit tests.

Any suggestions on how to get this to work would be appreciated.

Answer 1

I have a solution for when all else fails. Go to this website. http://www.lfd.uci.edu/~gohlke/pythonlibs/ This has ALMOST ALL Python libraries, with whl files for every type of installation, 32 bit and 64 bit, and all the Python versions since 2.6 (including beta 3.5)!