RSpec - 重新運行也被標記的失敗測試？

Question

我們已經編寫了大量的 selenium UI 測試，這些測試偶爾會因為 web 服務器太慢等愚蠢原因而失敗。 此外，我們大量使用 rspec 標簽來只運行我們想要運行的測試。 我們只想重試標記有某些內容且之前失敗的測試。 在下面的示例中，運行rspec --tag all --only_failures運行最后兩個測試，而我只希望它運行第二個測試。

spec_helper.rb

RSpec.configure do |config|
  config.example_status_persistence_file_path = "RunResults.txt"
end

測試規格.rb

require 'rspec'
require 'spec_helper'
require './test'

RSpec.describe Test do
    it "should pass", :all, :one do
        expect(1).to eql(1)
    end

    it "should fail", :all, :two do
        expect(1).to eql(2)
    end

    it "should fail", :three do
        expect(1).to eql(2)
    end
end

運行結果.txt

example_id               | status | run_time        |
------------------------ | ------ | --------------- |
./spec/test_spec.rb[1:1] | passed | 0.00055 seconds |
./spec/test_spec.rb[1:2] | failed | 0.01796 seconds |
./spec/test_spec.rb[1:3] | failed | 0.00017 seconds |

RSpec

$ rspec --tag all --only_failures
Run options: include {:all=>true, :last_run_status=>"failed"}
.FF

Failures:

  1) Test should fail
     Failure/Error: expect(1).to eql(2)
     
       expected: 2
            got: 1
     
       (compared using eql?)
     # ./spec/test_spec.rb:11:in `block (2 levels) in <top (required)>'

  2) Test should fail
     Failure/Error: expect(1).to eql(2)
     
       expected: 2
            got: 1
     
       (compared using eql?)
     # ./spec/test_spec.rb:15:in `block (2 levels) in <top (required)>'

Finished in 0.01949 seconds (files took 0.24733 seconds to load)
3 examples, 2 failures

Failed examples:

rspec ./spec/test_spec.rb:10 # Test should fail
rspec ./spec/test_spec.rb:14 # Test should fail

Answer 1

當你運行時：

rspec --tag all --only_failures

所有三個測試都被選中： 3 examples, 2 failures 。 所以它正在運行失敗的測試+標記為all的測試。

我希望我沒有錯過執行此操作的明顯命令。 我能想到的最好的辦法是將標簽保存到rspec持久性文件中的小猴子補丁：

config.example_status_persistence_file_path = "tmp/spec_persistence.txt"

require "rspec"
require "spec_helper"

# NOTE: override the way persistence file is saved;
#       just put this somewhere appropriate
module RSpec
  module Core
    class ExampleStatusPersister
      private
      def statuses_from_this_run
        @examples.map do |ex|
          result = ex.execution_result

          # NOTE: collect all the tags;
          #       maybe there is a better way, i haven't searched too hard
          tags = ex.metadata.except(*[:block, :description_args, :description, :full_description, :described_class, :file_path, :line_number, :location, :absolute_file_path, :rerun_file_path, :scoped_id, :execution_result, :example_group, :shared_group_inclusion_backtrace, :last_run_status])

          {
            :example_id => ex.id,
            :status     => result.status ? result.status.to_s : Configuration::UNKNOWN_STATUS,
            :run_time   => result.run_time ? Formatters::Helpers.format_duration(result.run_time) : "",

            # NOTE: save tags for reference later
            :tags       => tags.keys.join(","),
          }
        end
      end
    end
  end
end

RSpec.describe "Test" do
  it("1 should pass", :all, :one) { expect(1).to eql(1) }
  it("2 should fail", :all, :two) { expect(1).to eql(2) }
  it("3 should fail", :three)     { expect(1).to eql(2) }
end

https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L46

當您現在運行測試時，所有標簽都保存到持久性文件中：

$ cat tmp/spec_persistence.txt 
example_id         | status | run_time        | tags    |
------------------ | ------ | --------------- | ------- |
./spec/one.rb[1:1] | passed | 0.00075 seconds | one,all |
./spec/one.rb[1:2] | failed | 0.01538 seconds | two,all |
./spec/one.rb[1:3] | failed | 0.00021 seconds | three   |

這是對 select 進行適當測試的快速腳本：

# ./rspec_runner

#!/usr/bin/env ruby

require "rspec"

tag = ARGV[0]

# https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L8
results = RSpec::Core::ExampleStatusPersister.load_from("tmp/spec_persistence.txt")

filtered = results.filter_map do |result|
  result[:example_id] if result[:status] == "failed" && result[:tags].split(",").include?(tag)
end

if filtered.empty?
  puts "No failed tests found for tag: #{tag}"
else
  puts "Running failed tests with tag: #{tag}"
  puts "rspec #{filtered.join(' ')}"
  puts

  # NOTE: run tests by example_id
  system "rspec #{filtered.join(' ')}"
end

擊鼓：

$ ./rspec_runner all
Running failed tests with tag: all
rspec ./spec/one.rb[1:2]

Run options: include {:ids=>{"./spec/one.rb"=>["1:2"]}}
F
# ...
1 example, 1 failure
Failed examples:
rspec ./spec/one.rb:31 # Test 2 should fail

$ ./rspec_runner one
No failed tests found for tag: one