Engineering in Focus

An important quality of a software design is that it clearly expresses the concerns that it addresses and separates those concerns from one another. The great Robert C. Martin called this the Single Responsibility Principle. Among other reasons for following that principle is that when you need to change something in your program you only have to change it in one place.

The principle is just as important for tests: each test, and each layer of your testing architecture, must have a single job to do and express it clearly.

One important way in which that is true is with respect to test data. Most of the data in Fandor’s Rails application lives in ActiveRecord models. We usually test-drive features with Cucumber and drop down to RSpec to test complex areas in detail. Both Cucumber steps and RSpec examples use Factory Girl to create test instances of models.

Single-minded RSpec

In RSpec, then, the responsibility for each piece of data important to a test therefore belongs to either an example or in a factory definition or both. Since I started off by mentioning the Single Responsibility Principle, that word “both” should make you nervous. In fact “both” should never be the case: the responsibility for each piece of test data should either belong solely to its example or belong solely to the factory (meaning that it doesn’t matter in the test).

Here’s why: Suppose that we’re testing a method on our User class, can_watch_film?. Users have Roles which entail permissions. Here’s the spec

describe User do
  describe '#can_watch_films?' do
    it "returns true if the user can watch films" do
      create(:user).can_watch_films?.should be_true
    end
  end
end
 

and here’s the factory:

FactoryGirl.define do
  factory :user do
    sequence(:username) { |n| "username#{n}" }
    sequence(:first_name) { |n| "First#{n}" }
    sequence(:last_name) { |n| "Last#{n}" }
    # etc.
    roles { [ Role.find_by_name('subscriber') ] }
  end
end
 

Just thinking about the obvious second example you would write raises an issue: you can’t easily test that a User without the subscriber Role can’t watch films, because every User created by the factory has that Role. Suppose you’d written many examples which use this factory before realizing that you need some Users to not be subscribers. If you then changed the factory, you might need to change many examples too. Furthermore, think of a developer new to the application. There is nothing about the factory definition’s name to tell them whether the User is a subscriber or not, so they’ll have to figure that out (or to figure out that it doesn’t matter) from reading the rest of each example. Not very considerate of us, was it? The problem is that both the example and the factory have the responsibility of choosing the User’s Role.

The solution is simple: Give all of the responsibility to the example. Don’t add the Role in the factory definition; add it in the example:

describe User do
  describe '#can_watch_films?' do
    it "returns true if the user can watch films" do
      user = create :user, roles: [ Role.find_by_name('subscriber') ]
      user.can_watch_films?.should be_true
    end
  end
end
 

Now the intent of the example is completely clear, and it is less likely to need to change as you add more examples.

After you write more examples you might find yourself creating both subscribed and not-yet-subscribed users frequently. Define a factory for each:

FactoryGirl.define do
  factory :guest_user do
    sequence(:username) { |n| "username#{n}" }
    sequence(:first_name) { |n| "First#{n}" }
    sequence(:last_name) { |n| "Last#{n}" }
    # etc.
  end

  factory :subscribed_user, parent: :guest_user do
    roles { [ Role.find_by_name('subscriber') ] }
  end

end
 

These definitions are good because it is completely clear from their names what they do and do not entail, so new developers are more likely to use them correctly and examples will be clear:

describe User do
  describe '#can_watch_films?' do
    it "returns true if the user can watch films" do
      create(:subscribed_user).can_watch_films?.should be_true
    end

    it "returns false if the user can't watch films" do
      create(:guest_user).can_watch_films?.should be_false
    end

  end
end
 

Both the second version of our spec, where the example adds the Role, and the version we just showed, are good. The responsibility of choosing the User’s Roles is in one place, the example.

On-target Cucumber

In Cucumber, test data can be in any of three places: features, step definitions or factories, so there is even more opportunity to spread around responsibility for it. We’ve found two patterns for writing features which keep the responsibility entirely in the feature. First, a bad example to illustrate the problem:

Scenario: user visits a film's page
  Given there is a film
  When I go to the film's page
  Then I should see "Cucumber: The Movie"
 

This can only pass because the first step creates a film titled “Cucumber: The Movie”. If we ever need to change the title in the step, we’re going to have to change a lot of scenarios. Here’s a better way:

Scenario: user visits a film's page
  Given there is a film titled "Cucumber: The Movie"
  When I go to the film's page
  Then I should see "Cucumber: The Movie"
 

To be clear, here are the definitions of the first two steps:

Given /^there is a film titled "([^"]+?)"$/ do |title|
  @film = create :film, title: title
end

When /^I go to the film's page$/ do
  visit film_path(@film)
end
 

All good; the piece of data that we’re testing, the film’s title, only appears in the scenario. The :film factory definition might supply a default title, but we can ignore it.

What we just did is a bit verbose and repetitive, however. Instead, we could do it this way:

Scenario: user visits a film's page
  Given there is a film
  When I go to the film's page
  Then I should see the film's title
 

At the modest cost of writing an extra step (come to think of it, here it is),

Then /^I should see the film's title$/ do
  page.should have_content(@film.title)
end
 

we’ve removed incidental detail from the scenario. The responsibility for the film’s title is still in one place (that is, in one layer), but now it’s entirely in the steps and not in the scenario at all.

We’ve found that the second form is usually better, even though it requires you to write another step. It makes the scenario easier to read, and the extra step is often reused, so it doesn’t cost much anyway. And the second form is closer to what you should be doing anyway: writing steps that don’t refer to user interface specifics at all. However, in scenarios that create more than one test object of the same kind, you need the first form to distinguish one test object from another:

Scenario: user visits the list of all films
  Given there is a film titled "All About Everything"
  And there is a film titled "Zero Minutes To Go"
  When I go to the list of all films
  Then I should see "All About Everything" before "Zero Minutes To Go"
 

Although the designs of RSpec and Cucumber lead to different specific strategies for writing good tests in them, the overall principle is the same: put the responsibility for each piece of test data in exactly one place.

Empty out the factories

One more thought on factories: As seen above in the definition of the User factories, Factory Girl allows you to systematically vary model attributes using sequences. This is of course necessary when attribute values must be unique, but it also helps you police responsibility for test data: you can’t write an example or a scenario that asserts a value of some attribute that is generated by a sequence. Or, rather, if you do, the example or scenario will fail as soon as the order in which it is run changes and the sequence number is different. So an easy way to ensure that responsibility for test data is not divided between tests and factories is just to ensure that every attribute defined in a factory uses a sequence, never a meaningful value that someone might depend on in a test. Where that isn’t possible (as with an attribute that can only have certain values), choose the least surprising value you can, and if possible indicate that value in the name of the factory.