flask web server not starting in travis-ci in dual python environments

Question

I want to integrate my project into Travis-CI. My project involved a Flask web-server in Python with a postgresql database. All my tests use a web server running in debug mode. I'm trying to get this set up properly on Travis-CI and am having some troubles.

I'm building and testing under python 2.7 and 3.4, and the problem is my web server does not appear to start during the 3.4 build. The build for 2.7 starts first, runs fine, and all my tests pass, however the web server fails to start in the 3.4 build. When I run each build individually, both the 2.7 and 3.4 builds work fine and all tests pass. Is there a conflict between the web servers for each build? Here is my travis.yml file.

language: python

sudo: false

python:
- '2.7'
- '3.4'

os:
- linux

services:
- postgresql
- memcached

addons:
  postgresql: '9.4'
  apt:
    packages:
      xvfb
      nano

branches:
- master

before_install:
- wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O ~/miniconda.sh
- bash ~/miniconda.sh -b -p $HOME/miniconda
- export PATH="$HOME/miniconda/bin:$PATH"
- conda update --yes conda

install:
- conda create --yes -n test_env python=$TRAVIS_PYTHON_VERSION pip numpy scipy matplotlib ipython --quiet
- source activate test_env
- pip install -r requirements.txt --quiet
- pip install psycopg2 --quiet
- pip install blinker --quiet
- pip install nose --quiet
- pip install coverage --quiet
- pip install pytest --quiet
- pip install pytest-cov --quiet
- pip install pytest-sugar --quiet
- python setup.py install

before_script:
- export MANGA_LOCALHOST=1
- export PYTHONPATH=$PYTHONPATH:$TRAVIS_BUILD_DIR/python
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- sleep 3
- sh $TRAVIS_BUILD_DIR/bin/setup_travis
# make a local port based on python version, e.g. 5027 or 5034
- export LOCAL_MARVIN_PORT=50${TRAVIS_PYTHON_VERSION/.}
# start the local web server on port 50XX
- $TRAVIS_BUILD_DIR/bin/run_marvin -l -d -p $LOCAL_MARVIN_PORT > /dev/null &
- sleep 10
# list the processes running
- lsof -Pnl +M -i4
# test the web server is running
- curl http://localhost:$LOCAL_MARVIN_PORT/marvin2/api/general/getroutemap/

script:
- py.test python/marvin/tests -v --cov python/marvin --cov-report html

after_success:
- coveralls

Originally the local web server was running on port 5000, which I thought would be ok since each python build should be independent. It didn't work then and thought it might be related to the port, so I made each web server run on a unique port, but that hasn't fixed the problem. I'm printing various things as checks

The 2.7 build says the web server is running on 5027 and my curl produces output.

$ lsof -Pnl +M -i4
COMMAND  PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
Xvfb    3516     1000    1u  IPv4 67036287      0t0  TCP *:6099 (LISTEN)
python  3698     1000    5u  IPv4 67163206      0t0  TCP 127.0.0.1:5027 (LISTEN)
python  3698     1000    7u  IPv4 67154596      0t0  TCP 172.17.0.5:55988->155.101.19.34:443 (ESTABLISHED)
python  3698     1000    8u  IPv4 67162470      0t0  TCP 172.17.0.5:55964->155.101.19.34:80 (ESTABLISHED)
python  3713     1000    5u  IPv4 67163206      0t0  TCP 127.0.0.1:5027 (LISTEN)
python  3713     1000    7u  IPv4 67154596      0t0  TCP 172.17.0.5:55988->155.101.19.34:443 (ESTABLISHED)
python  3713     1000    8u  IPv4 67162470      0t0  TCP 172.17.0.5:55964->155.101.19.34:80 (ESTABLISHED)

$ curl http://localhost:$LOCAL_MARVIN_PORT/marvin2/api/general/getroutemap/
--------------------------------------------------------------------------------
INFO in __init__ [/home/travis/build/sdss/marvin/python/marvin/web/__init__.py:146]:
Loading config file: /home/travis/build/sdss/marvin/python/marvin/web/configuration/localhost.cfg
--------------------------------------------------------------------------------
WARNING:root: * CAUTION: flask-profiler is working without basic auth!
WARNING:werkzeug: * Debugger is active!
{
  "data": null, 
  "error": null, 
  "inconfig": {}, 
  "status": 1, 
  "traceback": null, 
  ...
}

It proceeds to go on and run all the tests. However, my 3.4 build seems to fail to start the web server on port 5034 with no indication why. The ports are indeed set differently.

$ lsof -Pnl +M -i4
COMMAND  PID     USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
Xvfb    3532     1000    1u  IPv4 34279725      0t0  TCP *:6099 (LISTEN)

$ curl http://localhost:$LOCAL_MARVIN_PORT/marvin2/api/general/getroutemap/
curl: (7) couldn't connect to host
The command "curl http://localhost:$LOCAL_MARVIN_PORT/marvin2/api/general/getroutemap/" failed and exited with 7 during .

Again, when I run each build individually, both 2.7 and 3.4 build successfully, run all tests and pass. Could it have something to do with how I'm dumping the standard out of the web server into /dev/null? Should I be doing something differently there? Is each build not exactly independent of one another?

Update: I just tried it without the /dev/null and got the same reesult. Also this time, the python 3.4 was ahead of the 2.7 build, and the 3.4 build successfully started the web server and ran the tests. The 2.7 build was unable to start the web server and was somehow blocked by the other build.

Answer 1

我通过将睡眠时间从10增加到20来解决了这个问题。显然，10秒太短了，无法启动Web服务器。