RSpec - 重新运行也被标记的失败测试？

Question

我们已经编写了大量的 selenium UI 测试，这些测试偶尔会因为 web 服务器太慢等愚蠢原因而失败。 此外，我们大量使用 rspec 标签来只运行我们想要运行的测试。 我们只想重试标记有某些内容且之前失败的测试。 在下面的示例中，运行rspec --tag all --only_failures运行最后两个测试，而我只希望它运行第二个测试。

spec_helper.rb

RSpec.configure do |config|
  config.example_status_persistence_file_path = "RunResults.txt"
end

测试规格.rb

require 'rspec'
require 'spec_helper'
require './test'

RSpec.describe Test do
    it "should pass", :all, :one do
        expect(1).to eql(1)
    end

    it "should fail", :all, :two do
        expect(1).to eql(2)
    end

    it "should fail", :three do
        expect(1).to eql(2)
    end
end

运行结果.txt

example_id               | status | run_time        |
------------------------ | ------ | --------------- |
./spec/test_spec.rb[1:1] | passed | 0.00055 seconds |
./spec/test_spec.rb[1:2] | failed | 0.01796 seconds |
./spec/test_spec.rb[1:3] | failed | 0.00017 seconds |

RSpec

$ rspec --tag all --only_failures
Run options: include {:all=>true, :last_run_status=>"failed"}
.FF

Failures:

  1) Test should fail
     Failure/Error: expect(1).to eql(2)
     
       expected: 2
            got: 1
     
       (compared using eql?)
     # ./spec/test_spec.rb:11:in `block (2 levels) in <top (required)>'

  2) Test should fail
     Failure/Error: expect(1).to eql(2)
     
       expected: 2
            got: 1
     
       (compared using eql?)
     # ./spec/test_spec.rb:15:in `block (2 levels) in <top (required)>'

Finished in 0.01949 seconds (files took 0.24733 seconds to load)
3 examples, 2 failures

Failed examples:

rspec ./spec/test_spec.rb:10 # Test should fail
rspec ./spec/test_spec.rb:14 # Test should fail

Answer 1

当你运行时：

rspec --tag all --only_failures

所有三个测试都被选中： 3 examples, 2 failures 。 所以它正在运行失败的测试+标记为all的测试。

我希望我没有错过执行此操作的明显命令。 我能想到的最好的办法是将标签保存到rspec持久性文件中的小猴子补丁：

config.example_status_persistence_file_path = "tmp/spec_persistence.txt"

require "rspec"
require "spec_helper"

# NOTE: override the way persistence file is saved;
#       just put this somewhere appropriate
module RSpec
  module Core
    class ExampleStatusPersister
      private
      def statuses_from_this_run
        @examples.map do |ex|
          result = ex.execution_result

          # NOTE: collect all the tags;
          #       maybe there is a better way, i haven't searched too hard
          tags = ex.metadata.except(*[:block, :description_args, :description, :full_description, :described_class, :file_path, :line_number, :location, :absolute_file_path, :rerun_file_path, :scoped_id, :execution_result, :example_group, :shared_group_inclusion_backtrace, :last_run_status])

          {
            :example_id => ex.id,
            :status     => result.status ? result.status.to_s : Configuration::UNKNOWN_STATUS,
            :run_time   => result.run_time ? Formatters::Helpers.format_duration(result.run_time) : "",

            # NOTE: save tags for reference later
            :tags       => tags.keys.join(","),
          }
        end
      end
    end
  end
end

RSpec.describe "Test" do
  it("1 should pass", :all, :one) { expect(1).to eql(1) }
  it("2 should fail", :all, :two) { expect(1).to eql(2) }
  it("3 should fail", :three)     { expect(1).to eql(2) }
end

https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L46

当您现在运行测试时，所有标签都保存到持久性文件中：

$ cat tmp/spec_persistence.txt 
example_id         | status | run_time        | tags    |
------------------ | ------ | --------------- | ------- |
./spec/one.rb[1:1] | passed | 0.00075 seconds | one,all |
./spec/one.rb[1:2] | failed | 0.01538 seconds | two,all |
./spec/one.rb[1:3] | failed | 0.00021 seconds | three   |

这是对 select 进行适当测试的快速脚本：

# ./rspec_runner

#!/usr/bin/env ruby

require "rspec"

tag = ARGV[0]

# https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L8
results = RSpec::Core::ExampleStatusPersister.load_from("tmp/spec_persistence.txt")

filtered = results.filter_map do |result|
  result[:example_id] if result[:status] == "failed" && result[:tags].split(",").include?(tag)
end

if filtered.empty?
  puts "No failed tests found for tag: #{tag}"
else
  puts "Running failed tests with tag: #{tag}"
  puts "rspec #{filtered.join(' ')}"
  puts

  # NOTE: run tests by example_id
  system "rspec #{filtered.join(' ')}"
end

击鼓：

$ ./rspec_runner all
Running failed tests with tag: all
rspec ./spec/one.rb[1:2]

Run options: include {:ids=>{"./spec/one.rb"=>["1:2"]}}
F
# ...
1 example, 1 failure
Failed examples:
rspec ./spec/one.rb:31 # Test 2 should fail

$ ./rspec_runner one
No failed tests found for tag: one