can I run SimpleCov in webrick?

Question

I have a rails app and a tester who is developing an external Selenium-based test system written in Java. We would like to know the current code-coverage of his test suite, so I would like to use SimpleCov in my :development env and then have him point his test suite at it.

I tried putting the SimpleCov lines in the top of env/development.rb, but it only created the ./coverage directory, it didn't actually update coverage reports for any of the clicks I tried in the application. I thought perhaps if I quit webrick it would write out then, but still nothing.

Can SimpleCov do this? (or any other coverage tool in Ruby?) How?

Answer 1

I got this working under Rails2 by doing the following (but Rails3 shouldn't be too different):

add the following lines to the top of your config/environments/development.rb :

require 'simplecov'

# clean the existing coverage directory before writing the new results.
Dir.foreach("coverage") do |file|
  next if [".",".."].include?(file)
  FileUtils.rm_rf(File.join("coverage",file), :verbose=>true)
end

SimpleCov.start 'rails' do
  add_filter "/vendor/"
end

Now, go and add a handy controller method to one of your controllers (I added it to an admin controller so I know it won't be accidentally called, but anything will do since you are guarding it for development only use):

class Admin::DebugController < ApplicationController

  if RAILS_ENV=="development"
    # see config/environments/development.rb
    def coverage
      SimpleCov.result.format!
      render :text => "Wrote results. STOP AND RESTART THE SERVER TO BEGIN A NEW COVERAGE RUN!!"
    end
  end

end

Now, start your rails server normally and let your external test suite run against it (or your manual integration QE tester ;)

When you are finished running through your integration tests, hit the url you setup in the admin controller, eg

http://localhost:3000/admin/debug/coverage

NOTE: this url only works/exists in development!

Voila! The coverage report will be written out to coverage/** in your rails directory. You can view it by loading coverage/index.html in your browser.

Justification

While the normal usage of SimpleCov is meant for unit and functional testing, adding the ability to evaluate external integration tests gives QE testers valuable feedback about what their testing strategies actually test within the target rails application.

This may be a trivial distinction for the majority of Rails applications, however it is useful when the application operates as a "pass-thru" for a large amount of data from other applications that is dynamically mixed into the application models (for example, a dashboard or console app). In these kinds of apps, rails testers may think they are testing permutations of the app, when in fact they are testing permutations of the underlying data (eg a network cluster). In this situation, code coverage gives an additional level of visibility into what is being tested.