Make chimp execution fail if running 0 tests

Question

Long story short, I have a meteor app passing in a continuous-integration pipeline which runs every tests before deploying. My tests run with chimp and I was installing chimp globally on my CI environment in each build with the latest version before running the tests.

recently, chimp made a somewhat major update which resulted in my chimp execution running 1..0 tests. After finding out it was because of the version of chimp, I made the change to the installation of chimp to be local and locked at a certain version.

The problem is that my pipeline was passing because hey, 0 tests passing is still 0 tests failing..!

I'm trying to make chimp fail if it runs no tests at all. How would be a good way to do it?

I tried greping the output and matching it to '1..0' resulting in a exit /b 1 status with no luck. The best solution would involve only the chimp command.

Thanks to anyone that could give me some hint on that.

Answer 1

Use the after test hook

You can put your code which runs after all tests in the after test hook (example below). I"m not sure on how to check check the number of tests run, but I think you can find out the same way the test reporter finds it. So, I'd suggest going through the source of one of the popular/simpler reporters like dot / spec and figuring it out from there.

describe('test', function() {
  after(function() { console.log('after'); });
  afterEach(function() { console.log('afterEach'); });

  it('fails sync', function(done) {
    after(function() { console.log('inner after 1'); });
    throw new Error('failed');
  });

  it('fails async', function(done) {
    after(function() { console.log('inner after 2'); });
    process.nextTick(function() {
      throw new Error('failed');
    });
  });
});

which produces the following output with mocha 1.1.12:

  ․afterEach
․afterEach
after
inner after 1
inner after 2


  0 passing (5 ms)
  2 failing

1) test fails sync:
 Error: failed
  at Context.<anonymous> (/private/tmp/so/test/main.js:7:11)
  at Test.Runnable.run (/private/tmp/so/node_modules/mocha/lib/runnable.js:194:15)
  at Runner.runTest (/private/tmp/so/node_modules/mocha/lib/runner.js:355:10)
  at /private/tmp/so/node_modules/mocha/lib/runner.js:401:12
  at next (/private/tmp/so/node_modules/mocha/lib/runner.js:281:14)
  at /private/tmp/so/node_modules/mocha/lib/runner.js:290:7
  at next (/private/tmp/so/node_modules/mocha/lib/runner.js:234:23)
  at Object._onImmediate (/private/tmp/so/node_modules/mocha/lib/runner.js:258:5)
  at processImmediate [as _immediateCallback] (timers.js:330:15)

2) test fails async:
 Error: failed
  at /private/tmp/so/test/main.js:13:12
  at process._tickCallback (node.js:415:13)

credits for example code from SO user Miroslav Bajtoš