Ruby unit testing frameworks comparison

As part of a session I presented at Ruby FunDay in ThoughtWorks Pune, I tried TDDing a simple rock-paper-scissors game. The frameworks I compared were rspec, expectations, and good ol' test/unit.

I'm hosting the code at bitbucket for reference sake: http://bitbucket.org/amanking/ruby-test-framework-comparison

(Note: I was demoing the use of test/unit live during the session and hence that implementation is incomplete. Plus, test/unit is a part of core Ruby and I expect most Ruby developers to be well-versed with it already.)

Here is the mind map that I drove my session with:

Overall, I continue to prefer the default choice of the Ruby and Rails community: RSpec. However, trying out Expectations, with its various restrictions, was a fresh and welcome change. It forces you to think about your design in a way that other testing frameworks may not always do. Jay Fields' attempt at coming up with a framework that promoted his ideas around unit testing is definitely worth a try.

Expectations

Here's a sample test using Expectations:

Expectations do
  expect :wins do
    rules = GameRules.say.rock.wins.against.scissor
    HandGesture.rock.against(HandGesture.scissor, :using => rules)
  end
 
  expect :loses do
    rules = GameRules.say.rock.wins.against.scissor
    HandGesture.scissor.against(HandGesture.rock, :using => rules)
  end
 
  expect :draw do
    HandGesture.scissor.against(HandGesture.scissor)
  end
  # ...
end

In summary, I liked that Expectations restricts us to one assertion at a time. Regarding expecting literals, I like that it discourages unnecessary variables for expected values... but I'm afraid that it'll probably push some people to indulge in Primitive Obsession. If you're careful not to stray though, I like that it'll make you think of Value Objects. But again here, I'd be wary of not making Entity Objects into Value Objects just for testing purposes.

Funnily, when I TDD'ed using Expecations I came up with a DSLish approach to specifying game rules, which I did not arrive at when I TDD'ed using RSpec or Test::Unit. Of course, this may be just a fluke but perhaps deliberately focusing on small tests will make you think along those lines.

RSpec

As for RSpec, I enjoy its readability mostly, and that it helps focus on scenarios rather than just methods. I also appreciate the ability to have nested describe blocks (and hence, nested setup/teardown) but I'd be wary of overusing the same: although the auto-generated specification documentation will turn out pretty with nested describes, maintaining the test suite will became a pain over time.

I also like the shared examples concept in RSpec which allows reuse of assertions in different scenarios.

Here are some specs:

describe Game do
  before(:each) do
    @game = Game.new
  end
 
  it "should not be playable after a play" do
    @game.play
    lambda { @game.play }.should raise_error('Already played')
  end
 
  describe "with one player" do
    before(:each) do
      @player = Player.new('aman', Rock.new)
      @game.register(@player)
    end
 
    it "should return surviving players on play" do
      @game.play.should == [@player]
    end
  end
 
  describe "with two players with same gestures" do
    before(:each) do
      @rock_player = Player.new('rocker', Rock.new)
      @another_rock_player = Player.new('rocker', Rock.new)
 
      [@rock_player, @another_rock_player].each {|player| @game.register(player) }
    end
 
    it "should signal draw on play" do
      lambda { @game.play }.should raise_error(Game::Draw)
    end
  end
  # ...
end

Here's an example of shared example:

sshared_examples_for "hand gesture" do
  it "should lose" do
    @subject_class.new.against(@loses_against_class.new).should == :loses
  end
 
  it "should win" do
    @subject_class.new.against(@wins_against_class.new).should == :wins
  end
 
  it "should draw" do
    @subject_class.new.against(@subject_class.new).should == :draw
  end
end
 
describe Rock do
  before(:each) do
    @subject_class = Rock
    @loses_against_class = Paper
    @wins_against_class = Scissor
  end
 
  it_should_behave_like "hand gesture"
end
 
describe Paper do
  before(:each) do
    @subject_class = Paper
    @loses_against_class = Scissor
    @wins_against_class = Rock
  end
 
  it_should_behave_like "hand gesture"
end
 
# ...

Tags: technology:ruby, technology:coding Last modified 08:47 Thu, 10 Sept 2009 by AmanKing. Accessed 250 times Children What Links Here Share Except where expressly noted, this work is licensed under a Creative Commons License.