[英]RSpec - Rerun failed tests that are also tagged?
We have written a large number of selenium UI tests that very occasionally fail due to dumb reasons like a web server being too slow.我们已经编写了大量的 selenium UI 测试,这些测试偶尔会因为 web 服务器太慢等愚蠢原因而失败。 Additionally, we make heavy use of rspec tags to only run the tests that we want to run.
此外,我们大量使用 rspec 标签来只运行我们想要运行的测试。 We want to retry ONLY the tests that are tagged with something AND that have previously failed.
我们只想重试标记有某些内容且之前失败的测试。 In the following example, running
rspec --tag all --only_failures
runs the last two tests, when I only want it to run the 2nd test.在下面的示例中,运行
rspec --tag all --only_failures
运行最后两个测试,而我只希望它运行第二个测试。
spec_helper.rb spec_helper.rb
RSpec.configure do |config|
config.example_status_persistence_file_path = "RunResults.txt"
end
test_spec.rb测试规格.rb
require 'rspec'
require 'spec_helper'
require './test'
RSpec.describe Test do
it "should pass", :all, :one do
expect(1).to eql(1)
end
it "should fail", :all, :two do
expect(1).to eql(2)
end
it "should fail", :three do
expect(1).to eql(2)
end
end
RunResults.txt运行结果.txt
example_id | status | run_time |
------------------------ | ------ | --------------- |
./spec/test_spec.rb[1:1] | passed | 0.00055 seconds |
./spec/test_spec.rb[1:2] | failed | 0.01796 seconds |
./spec/test_spec.rb[1:3] | failed | 0.00017 seconds |
RSpec RSpec
$ rspec --tag all --only_failures
Run options: include {:all=>true, :last_run_status=>"failed"}
.FF
Failures:
1) Test should fail
Failure/Error: expect(1).to eql(2)
expected: 2
got: 1
(compared using eql?)
# ./spec/test_spec.rb:11:in `block (2 levels) in <top (required)>'
2) Test should fail
Failure/Error: expect(1).to eql(2)
expected: 2
got: 1
(compared using eql?)
# ./spec/test_spec.rb:15:in `block (2 levels) in <top (required)>'
Finished in 0.01949 seconds (files took 0.24733 seconds to load)
3 examples, 2 failures
Failed examples:
rspec ./spec/test_spec.rb:10 # Test should fail
rspec ./spec/test_spec.rb:14 # Test should fail
When you run:当你运行时:
rspec --tag all --only_failures
all three tests get selected: 3 examples, 2 failures
.所有三个测试都被选中:
3 examples, 2 failures
。 So it's running failed tests + tests tagged all
.所以它正在运行失败的测试+标记为
all
的测试。
I hope I didn't miss an obvious command for doing this.我希望我没有错过执行此操作的明显命令。 Best I could come up is a little monkey patch to save tags into rspec persistence file:
我能想到的最好的办法是将标签保存到rspec持久性文件中的小猴子补丁:
config.example_status_persistence_file_path = "tmp/spec_persistence.txt"
require "rspec"
require "spec_helper"
# NOTE: override the way persistence file is saved;
# just put this somewhere appropriate
module RSpec
module Core
class ExampleStatusPersister
private
def statuses_from_this_run
@examples.map do |ex|
result = ex.execution_result
# NOTE: collect all the tags;
# maybe there is a better way, i haven't searched too hard
tags = ex.metadata.except(*[:block, :description_args, :description, :full_description, :described_class, :file_path, :line_number, :location, :absolute_file_path, :rerun_file_path, :scoped_id, :execution_result, :example_group, :shared_group_inclusion_backtrace, :last_run_status])
{
:example_id => ex.id,
:status => result.status ? result.status.to_s : Configuration::UNKNOWN_STATUS,
:run_time => result.run_time ? Formatters::Helpers.format_duration(result.run_time) : "",
# NOTE: save tags for reference later
:tags => tags.keys.join(","),
}
end
end
end
end
end
RSpec.describe "Test" do
it("1 should pass", :all, :one) { expect(1).to eql(1) }
it("2 should fail", :all, :two) { expect(1).to eql(2) }
it("3 should fail", :three) { expect(1).to eql(2) }
end
https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L46 https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L46
When you run tests now, all tags are saved into persistence file:当您现在运行测试时,所有标签都保存到持久性文件中:
$ cat tmp/spec_persistence.txt
example_id | status | run_time | tags |
------------------ | ------ | --------------- | ------- |
./spec/one.rb[1:1] | passed | 0.00075 seconds | one,all |
./spec/one.rb[1:2] | failed | 0.01538 seconds | two,all |
./spec/one.rb[1:3] | failed | 0.00021 seconds | three |
This is a quick script to select the appropriate tests:这是对 select 进行适当测试的快速脚本:
# ./rspec_runner
#!/usr/bin/env ruby
require "rspec"
tag = ARGV[0]
# https://github.com/rspec/rspec-core/blob/v3.12.0/lib/rspec/core/example_status_persister.rb#L8
results = RSpec::Core::ExampleStatusPersister.load_from("tmp/spec_persistence.txt")
filtered = results.filter_map do |result|
result[:example_id] if result[:status] == "failed" && result[:tags].split(",").include?(tag)
end
if filtered.empty?
puts "No failed tests found for tag: #{tag}"
else
puts "Running failed tests with tag: #{tag}"
puts "rspec #{filtered.join(' ')}"
puts
# NOTE: run tests by example_id
system "rspec #{filtered.join(' ')}"
end
Drumroll:击鼓:
$ ./rspec_runner all
Running failed tests with tag: all
rspec ./spec/one.rb[1:2]
Run options: include {:ids=>{"./spec/one.rb"=>["1:2"]}}
F
# ...
1 example, 1 failure
Failed examples:
rspec ./spec/one.rb:31 # Test 2 should fail
$ ./rspec_runner one
No failed tests found for tag: one
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.