Rspec 使用 --only-failures 重新运行所有测试，甚至通过测试

Question

我们有一个 Ruby gem，它包装了 rspec 用于我们的 selenium 测试。 我试图通过重新运行以前运行的失败来实现重试逻辑，但在第二次运行时它只是重试所有内容，我认为这是因为它没有在第二次运行时加载 spec_helper，其中包含example_status_persistence_file_path位置。

我在 spec_helper 中添加了一个调试puts语句，你会看到它没有被调用两次，只有一次。 但是，我也没有收到未设置example_status_persistence_file_path时出现的错误，所以我不知道该怎么做。

spec_helper.rb

RSpec.configure do |config|
  config.example_status_persistence_file_path = "logs/LO_RunResults.txt"
end

my_spec.rb

it "should pass" do
  expect(true).to be true
end

it "should fail" do
  expect(false).to be true
end

rspec_runner.rb文件

test_results_file = "#{File.expand_path(Dir.pwd)}/logs/RunResults.txt"
File.delete(test_results_file) if File.exist?(test_results_file)

args = %w(-r spec_helper --order defined --format CustomFormatter --format documentation)
RSpec::Core::Runner.run(args, $stderr, $stdout)

# RunResults.txt is generated from previous run

args = %w(-r spec_helper --order defined --format CustomFormatter --format documentation --only-failures)
RSpec::Core::Runner.run(args, $stderr, $stdout)

当我运行ruby rspec_runner.rb ，这是我的 output：

#################### Running spec_helper ####################

Test Suite - my_spec
  Positive Tests
    should pass
    should fail (FAILED - 1)

Failures:

  1) Test Suite - my_spec Positive Tests should fail
     Failure/Error: expect(false).to be true
     
       expected true
            got false
     # ./spec/my_spec.rb:59:in `block (3 levels) in <top (required)>'

Finished in 0.06265 seconds (files took 0.71696 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/my_spec.rb:58 # Test Suite - my_spec Positive Tests should fail

Run options: include {:last_run_status=>"failed"}

Test Suite - my_spec
  Positive Tests
    should pass
    should fail (FAILED - 2)

Failures:

  1) Test Suite - my_spec Positive Tests should fail
     Got 2 failures:

     1.1) Failure/Error: expect(false).to be true
          
            expected true
                 got false
          # ./spec/my_spec.rb:59:in `block (3 levels) in <top (required)>'

     1.2) Failure/Error: expect(false).to be true
          
            expected true
                 got false
          # ./spec/my_spec.rb:59:in `block (3 levels) in <top (required)>'

  2) Test Suite - my_spec Positive Tests should fail
     Got 2 failures:

     2.1) Failure/Error: expect(false).to be true
          
            expected true
                 got false
          # ./spec/my_spec.rb:59:in `block (3 levels) in <top (required)>'

     2.2) Failure/Error: expect(false).to be true
          
            expected true
                 got false
          # ./spec/my_spec.rb:59:in `block (3 levels) in <top (required)>'

Finished in 0.03786 seconds (files took 0.79882 seconds to load)
4 examples, 2 failures

Failed examples:

rspec ./spec/my_spec.rb:58 # Test Suite - my_spec Positive Tests should fail
rspec ./spec/my_spec.rb:58 # Test Suite - my_spec Positive Tests should fail


Process finished with exit code 0

Answer 1

您可以尝试在两次运行之间使用RSpec.clear_examples以强制 RSpec 重新加载示例：

test_results_file = "#{File.expand_path(Dir.pwd)}/logs/RunResults.txt"
File.delete(test_results_file) if File.exist?(test_results_file)

args = %w(-r spec_helper --order defined --format CustomFormatter --format documentation)
RSpec::Core::Runner.run(args, $stderr, $stdout)

RSpec.clear_examples

# RunResults.txt is generated from previous run

args = %w(-r spec_helper --order defined --format CustomFormatter --format documentation --only-failures)
RSpec::Core::Runner.run(args, $stderr, $stdout)

文档在这里供参考： https://www.rubydoc.info/github/rspec/rspec-core/RSpec.clear_examples 。

用于确保示例在同一进程中的多次运行之间重新加载，并确保用户配置得以保留。

如果用户想清除所有示例但在同一进程中多次使用运行器时保留当前配置，则必须调用此方法。