在Ruby中重新运行失败的单元测试

Question

I have some unit tests written using Test::Unit::TestCase, with XML generated by ci_reporter. However, due to circumstances beyond my control, they may occasionally fluctuate, and randomly fail. I'd like to detect when a test fails, and attempt to re-run it.

I tried doing this by monkey-patching 'teardown' to check 'passed?', and re-running the tests on a failure. However, the XML output will still show the first failed case, and not the second (now passing) run.

Answer 1

This sounds a bit like the opposite of Multiple tests with minitest

Perhaps this is a possibility: Copy your test case in an own file. As an example, try the following test:

#store it as file 'testcase.rb'
gem 'test-unit'
require 'test/unit'

class X < Test::Unit::TestCase
  def test_1
    num = rand(10) 
    assert_true( num < 2, "Value is #{num}")
  end
end 

Then define your test call in a rake task:

require 'rake'

TEST_REPETION = 10
task :test do
  success = false
  TEST_REPETION.times{
    stdout = `ruby testcase.rb`

    if stdout =~ /Failure/
      puts "Failure occured - redo the test"
    else
      puts 'Tests ok'
      success = true
      exit 
    end
  }

  puts "Stopped after #{TEST_REPETION} tries" unless success
end

Now the test is called, until the test succeed or TEST_REPETION are done.

Remarks:

• Rake isn't needed, you may do the call without rake ( My template was a rake task)
• This works only, if your xml changes for each run (it must be regenerated before the test. else you test always the same).
• You may store the test result ( stdout ) in a file and use it later to analyze, which tests failed and try to retest them.