Travis script exit code never fails even when it should

Question

The script portion of my Travis yml file that looks like this:

script:
- ./run_tests.sh

The script itself runs some tests on Sauce Labs. If the script fails due to test failures, it still exits with code 0 and the build continues on to pass as well. Why doesn't the script exit with a failure code if a test fails?

When I output the exit code from the end of my script file, I get 0. When I output the exit code in the .travis.yml file immediately after the script command, I get 1:

echo $?
0
The command "./run_tests.sh" exited with 0.
$ echo $?
1

Answer 1

I realized this was because I'm actually running my tests using unittest.TextTestRunner, and the exit code from those tests is always 0 unless you specifically catch the test failures and exit based on them:

ret = not runner.run(test_suite).wasSuccessful()
sys.exit(ret)