Cannot install lxml via pip

Question

I want to install lxml via pip, but it just stops, without any error or sth. When I execute

pip install lxml -vvv

i see, that it stops in the line:

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/libxml2 -Isrc/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-armv6l-2.7/src/lxml/lxml.etree.o -w

I can leave the console for an hour. Nothings happens. No error message. Nothing.

PS: Yes,

libxml2-dev libxslt-dev python-dev

are all installed.

Answer 1

For all people finding that question with the same problem: Compiling processes are very slow on the Raspberry Pi. It just doesn't have enough power. Just don't do that, it's no fun.

Answer 2

Why that didn't work is very simple. Compiling Python's lxml package takes just a little more RAM than you have allocated for it. The default swap space of 100MB plus 512MB RAM means that gcc and kswapd literally fight against each other. It will probably stay at lxml.etree.c until the end of days.

How to proceed:

First we identify what is consuming a lot of memory

ps aux | head -1; ps aux | sort -rnk 4 | head -5

and temporarily terminate corresponding demons and processes. After that, we expand the swap file to 2GB.

sudo dphys-swapfile swapoff

to turn swapping off. Then edit /etc/dphys-swapfile and modify the variable CONF_SWAPSIZE :

CONF_SWAPSIZE = 2048

run

dphys-swapfile setup

which will create and initialize the file.

sudo dphys-swapfile swapon

to turn swapping on again.

---

I needed this for Home Assistant as it calls for Python 3.8 and that is not in the package feed.

On my Raspberry Pi Zero

pip3 install lxml

took approx.70min.