Unable to setup “sphinx” on gitlab-ci

Question

Unable to setup sphinx on gitlab-ci

I'm trying to setup and run sphnix on gitlab-ci. It's installed on my local machine and server works fine. But not at gitlab-ci

Here's a part of my gitlab-ci:

  before_script:
    - apt-get update -qq && apt-get install -yqq [......] python-pip
    - pip install sphinx

    # ..........
    - bundle exec rake ts:index
    - bundle exec rake ts:start

The output of installing sphinx:

$ pip install sphinx
Downloading/unpacking sphinx
[................]
Installing collected packages: sphinx, babel, [.............]
Compiling /tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncfilters.py ...
  File "/tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncfilters.py", line 7
    async def auto_to_seq(value):
            ^
SyntaxError: invalid syntax

Compiling /tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncsupport.py ...
  File "/tmp/pip-build-V2Tzes/Jinja2/jinja2/asyncsupport.py", line 22
    async def concat_async(async_gen):
            ^
SyntaxError: invalid syntax

  Running setup.py install for MarkupSafe

    building 'markupsafe._speedups' extension
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c markupsafe/_speedups.c -o build/temp.linux-x86_64-2.7/markupsafe/_speedups.o
    markupsafe/_speedups.c:12:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    ==========================================================================
    WARNING: The C extension could not be compiled, speedups are not enabled.
    Failure information, if any, is above.
    Retrying the build without the C extension now.


    ==========================================================================
    WARNING: The C extension could not be compiled, speedups are not enabled.
    Plain-Python installation succeeded.
    ==========================================================================
Successfully installed sphinx babel sphinxcontrib-websupport Jinja2 alabaster imagesize Pygments snowballstemmer docutils packaging typing pytz MarkupSafe pyparsing
Cleaning up...

As you can see, it's successeded.

However, when it gets to this point, it fails:

  $ bundle exec rake ts:index
  Generating configuration to /builds/my_user/my_ap123/config/test.sphinx.conf
  sh: 1: indexer: not found

  The Sphinx indexing command failed:
    Command: indexer --config "/builds/my_user/my_ap123/config/test.sphinx.conf" --all
    Status:  127
    Output:  See above

Why is that? How to fix it?

Answer 1

pip install sphinx did not fully succeed. during installation, it is complaining about missing Python.h

fatal error: Python.h: No such file or directory

You need to install a development package of python as well. For python3x use

sudo apt-get install python3-dev

For python2x use

sudo apt-get install python-dev

You can add it to before_script

before_script:
    - apt-get update -qq && apt-get install -yqq [......] python-pip python3-dev
    - pip install sphinx

Update :

To install sphinx search engine using gitlab ci update your before_script as follow

before_script:
    - add-apt-repository ppa:builds/sphinxsearch-rel22 sphinxsearch
    - apt-get update -qq && apt-get install -yqq [......] python-pip python3-dev mysql-client unixodbc libpq5 
    - pip install sphinxsearch