When and when not to stub/mock a test

Question

I am making a concerted effort to wrap my head around Rspec in order to move towards more of a TDD/BDD development pattern. However, I'm a long way off and struggling with some of the fundamentals:

Like, when exactly should I be using mocks/stubs and when shouldn't I?

Take for example this scenario: I have a Site model that has_many :blogs and the Blog model has_many :articles . In my Site model I have a callback filter that creates a default set of blogs and articles for every new site. I want to test that code, so here goes:

describe Site, "when created" do

  include SiteSpecHelper

  before(:each) do
    @site = Site.create valid_site_attributes
  end

  it "should have 2 blogs" do
    @site.should have(2).blogs
  end

  it "should have 1 main blog article" do
    @site.blogs.find_by_slug("main").should have(1).articles
  end

  it "should have 2 secondary blog articles" do
    @site.blogs.find_by_slug("secondary").should have(2).articles
  end

end

Now, if I run that test, everything passes. However, it's also pretty slow as it's creating a new Site, two new Blogs and three new Articles - for every single test! So I wonder, is this a good candidate for using stubs? Let's give it a go:

describe Site, "when created" do

  include SiteSpecHelper

  before(:each) do
    site = Site.new
    @blog = Blog.new
    @article = Article.new
    Site.stub!(:create).and_return(site)
    Blog.stub!(:create).and_return(@blog)
    Article.stub!(:create).and_return(@article)
    @site = Site.create valid_site_attributes
  end

  it "should have 2 blogs" do
    @site.stub!(:blogs).and_return([@blog, @blog])
    @site.should have(2).blogs
  end

  it "should have 1 main blog article" do
    @blog.stub!(:articles).and_return([@article])
    @site.stub_chain(:blogs, :find_by_slug).with("main").and_return(@blog)
    @site.blogs.find_by_slug("main").should have(1).articles
  end

  it "should have 2 secondary blog articles" do
    @blog.stub!(:articles).and_return([@article, @article])
    @site.stub_chain(:blogs, :find_by_slug).with("secondary").and_return(@blog)
    @site.blogs.find_by_slug("secondary").should have(2).articles
  end

end

Now all the tests still pass, and things are a bit speedier too. But, I've doubled the length of my tests and the whole exercise just strikes me as utterly pointless, because I'm no longer testing my code, I'm just testing my tests.

Now, either I've completely missed the point of mocks/stubs, or I'm approaching it fundamentally wrong, but I'm hoping someone might be able to either:

• Improve me tests above so it uses stubs or mocks in a way that actually tests my code, rather than my tests.
• Or, tell me if I should even be using stubs here - or whether in fact this is completely unnecessary and I should be writing these models to the test database.

Answer 1

But, I've doubled the length of my tests and the whole exercise just strikes me as utterly pointless, because I'm no longer testing my code, I'm just testing my tests.

This is the key right here. Tests that don't test your code aren't useful. If you can negatively change the code that your tests are supposed to be testing, and the tests don't fail, they're not worth having.

As a rule of thumb, I don't like to mock/stub anything unless I have to. For example, when I'm writing a controller test, and I want to make sure that the appropriate action happens when a record fails to save, I find it easier to stub the object's save method to return false, rather than carefully crafting parameters just so in order to make sure a model fails to save.

Another example is for a helper called admin? that just returns true or false based on whether or not the currently logged-in user is an admin or not. I didn't want to go through faking a user login, so I did this:

# helper
def admin?
  unless current_user.nil?
    return current_user.is_admin?
  else
    return false
  end
end

# spec
describe "#admin?" do
  it "should return false if no user is logged in" do
    stubs(:current_user).returns(nil)
    admin?.should be_false
  end

  it "should return false if the current user is not an admin" do
    stubs(:current_user).returns(mock(:is_admin? => false))
    admin?.should be_false
  end

  it "should return true if the current user is an admin" do
    stubs(:current_user).returns(mock(:is_admin? => true))
    admin?.should be_true
  end
end

As a middle ground, you might want to look into Shoulda . This way you can just make sure your models have an association defined , and trust that Rails is well-tested enough that the association will "just work" without you having to create an associated model and then counting it.

I've got a model called Member that basically everything in my app is related to. It has 10 associations defined. I could test each of those associations, or I could just do this:

it { should have_many(:achievements).through(:completed_achievements) }
it { should have_many(:attendees).dependent(:destroy) }
it { should have_many(:completed_achievements).dependent(:destroy) }
it { should have_many(:loots).dependent(:nullify) }
it { should have_one(:last_loot) }
it { should have_many(:punishments).dependent(:destroy) }
it { should have_many(:raids).through(:attendees) }
it { should belong_to(:rank) }
it { should belong_to(:user) }
it { should have_many(:wishlists).dependent(:destroy) }

Answer 2

This is exactly why I use stubs/mocks very rarely (really only when I'm going to be hitting an external webservice). The time saved just isn't worth the added complexity.

There are better ways to speed up your testing time, and Nick Gauthier gives a good talk covering a bunch of them - see the video and the slides .

Also, I think a good option is to try out an in-memory sqlite database for your test runs. This should cut down on your database time by quite a bit by not having to hit the disk for everything. I haven't tried this myself, though (I primarily use MongoDB, which has the same benefit), so tread carefully. Here's a fairly recent blog post on it.

Answer 3

I'm not so sure with agreeing on the others. The real problem (as I see it) here, is that you're testing multiple pieces of interesting behavior with the same tests (the finding behavior, and the creation). For reasons on why this is bad, see this talk: http://www.infoq.com/presentations/integration-tests-scam . I'm assuming for the rest of this answer that you want to test that creation is what you want to test.

Isolationist tests often seem unwieldy, but that's often because they have design lessons to teach you. Below are some basic things I can see out of this (though without seeing the production code, I can't do too much good).

For starters, to query the design, does having the Site add articles to a blog make sense? What about a class method on Blog called something like Blog.with_one_article . This then means all you have to test is that that class method has been called twice (if [as I understand it for now], you have a "primary" and "secondary" Blog for each Site , and that the associations are set up (I haven't found a great way to do this in rails yet, I usually don't test it).

Furthermore, are you overriding ActiveRecord's create method when you call Site.create ? If so, I'd suggest making a new class method on Site named something else ( Site.with_default_blogs possibly?). This is just a general habit of mine, overriding stuff generally causes problems later on in projects.