Transcript Testing/Testing In Rails 1 - SE-Wiki

Testing/Testing In Rails 1
Alan and Saskia
2/8/2008
Outline
•
•
•
•
•
•
Testing In General
Unit Testing and Test First!
Integration Testing
Functional Testing
Fixture
Mocking with Mocha
Testing In General (1)
• Formal Definition
– Testing is the process of finding differences
between the expected behavior specified by
system models and the observed behavior of the
implemented system.
– The goal is to design tests that exercise defects in
the system and to reveal problems
Testing In General (2)
• Alternative Definition
– Testing has to demonstrate that faults are not
present at all.
• Almost impossible to show
• May lead to the selection of test data that have a low
probability of causing the program to fail
• Even more alternative (Wilson Bilkovich)
– Writing applications without tests makes you a
bad person, incapable of love.
Types of Testing (1)
Types of Testing (2)
Unit Testing in Ruby
• Unit Testing
– Testing small units of codes (normally methods)
• Test::Unit
– Ruby’s built-in testing framework
– xUnit family (Java’s Junit, .NET’s Nunit)
• Concepts
– Assertions: comparison of expected value and result of an
expression
– Failure: assertion failure
– Error: exception or runtime error
– Green vs red bar
Unit Testing in Ruby (2)
require 'roman'
<<Class inclusion
require 'test/unit'
class TestRoman < Test::Unit::TestCase <<super class
def setup
<<code to be run before all test methods
end
<<naming convention
def test_simple
assert_equal("i", Roman.new(1).to_s)
<<assertions
assert_equal("ix", Roman.new(9).to_s)
end
def teardown
end
end
<<code to be run after all test methods
Assertions (1)
• assert(boolean, [ message ] )
– Fails if boolean is false or nil.
• assert_nil(obj, [ message ] )
• assert_not_nil(obj, [ message ] )
– Expects obj to be (not) nil.
• assert_equal(expected, actual, [ message ] )
• assert_not_equal(expected, actual, [ message ] )
– Expects obj to equal/not equal expected, using ==.
• assert_in_delta(expected_float, actual_float, delta, [ message
])
– Expects that the actual floating-point value is within delta
of the expected value.
Assertions (2)
• assert_raise(Exception, . . . ) { block }
• assert_nothing_raised(Exception, . . . ) { block }
– Expects the block to (not) raise one of the listed exceptions.
• assert_instance_of(klass, obj, [ message ] )
• assert_kind_of(klass, obj, [ message ] )
– Expects obj to be a kind/instance of klass.
• assert_respond_to(obj,message, [ message ] )
– Expects obj to respond to message (a symbol).
• assert_match(regexp, string, [ message ] )
• assert_no_match(regexp, string, [ message ] )
– Expects string to (not) match regexp.
• assert_same(expected, actual, [ message ] )
• assert_not_same(expected, actual, [ message ] )
– Expects expected.equal?(actual).
Assertions (3)
• assert_operator(obj1, operator, obj2, [ message ] )
– Expects the result of sending the message operator to obj1 with
parameter obj2 to be true.
• assert_throws(expected_symbol, [ message ] ) { block }
– Expects the block to throw the given symbol.
• assert_send(send_array, [ message ] )
– Sends the message in send_array[1] to the receiver in send_array[0],
passing the rest of send_array as arguments. Expects the return value
to be true.
• flunk(message="Flunked")
– Always fail.
Test First!
• What is it about?
– Write the test cases first. Then write minimal
codes to pass the tests
– Write more tests. Write more minimal codes to
pass the tests
– Iterate the process till all functional requirements
are satisfied 
Test First Guidelines
• The name of the test should describe the requirement of the code
• There should be at least one test for each requirement of the code. Each
possible path through of the code is a different requirement
• Only write the simplest possible code to get the test to pass, if you know
this code to be incomplete, write another test that demonstrates what
else the code needs to do
• A test should be similar to sample code, in that it should be clear to
someone unfamiliar with the code as to how the code is intended to be
used
• If a test seems too large, see if you can break it down into smaller tests
• If you seem to be writing a lot of code for one little test, see if there are
other related tests you could write first, that would not require as much
code
• More at http://www.xprogramming.com/xpmag/testFirstGuidelines.htm
Test First Demo
Unit Testing in Rails
• In rails, unit testing is geared towards testing
of individual functions created in a model
• For each model generated, a test file is
automatically created in tests/unit directory
• script/generate model product name:string description:string
exists app/models/
exists test/unit/
exists test/fixtures/
create app/models/product.rb
create test/unit/product_test.rb
….
Testing/Testing In Rails 1
Saskia
Outline
•
•
•
•
•
•
Testing In General
Unit Testing and Test First!
Integration Testing
Functional Testing
Fixture
Mocking with Mocha
Types of Testing (1)
Integration Testing in RoR
• Integration Testing
– story-level, tests interactions & interfaces of various actions
supported by the application, across all controllers
– Find bugs with session management and routing, triggered
by certain cruft accumulating in a user’s session
• In 'test/integration':
– Create test file with 'script/generate integration_test stories_test'
– Class IntegrationTest inherits from Test::Unit
Integration Testing in RoR
• Story set describes how the application ought to function
– Interfaces: exchange of right data
– Interaction: employment of right components
• Example: signing up, getting account, multiple users
• Testing time grows with number of integrated units
– Bottom-Up: Integrate tested Units to subsystems as new
components
Integration Testing in RoR
File 'stories_test.rb' in 'test/integration':
require "#{File.dirname(__FILE__)}/../test_helper" <<class inclusion
class StoriesTest < ActionController::IntegrationTest <<Integration Test Class
fixtures :users, :accounts
def test_stories
<<naming convention
get "/signup"
assert_response :success
<<assertions
assert_template "signup/index"
post "/signup", :name => "Bob", :user_name => "bob",
:password => "secret"
assert_response :redirect
follow_redirect!
assert_response :success
assert_template "account/index"
end
Integration Testing in Ruby (2)
Run by: rake test_integration or ruby stories_test.rb
Output:
Started
<<Each spot represents a test, sorted alphabetically,
.F..E
3 values:
. means successful test (pass)
Finished in 0.01 seconds.
F means failed test
E means an error occurred
1) Failure:
where/which test
what went wrong
2) Error:
where/which test
error_type
what went wrong
<<Listing of failures & errors:
Details on where and what
Moves on with first wrong assertion.
5 tests, 9 assertions, 1 failure, 1 error.
Types of Testing (1)
Functional Testing in Ruby
• Functional Testing
– single controllers and interactions between the
models it employs
• In directory 'test/function':
– 'functional_controller_test.rb' stubs for each 'script/generate
controller'
Functional Testing in RoR
require File.dirname(__FILE__) + '/../test_helper'
require 'home_controller'
<< grab HomeController for testing
class HomeControllerTest < Test::Unit::TestCase
def setup
<< setup of 3 typical funtional Test objects:
* controller to be tested
@controller = HomeController.new
* TestRequest to
@request = ActionController::TestRequest.new simulate web request
@response = ActionController::TestResponse.new * TestResponse to provide
information about test request
end
def test_index << test of main index page
<< get method simulates request on action called index
get :index
assert_response. :success << assertion assures that request successful
end
5 request types supported as methods in Rails:
end
get, post, put, head, delete
Fixtures in RoR
• Fixtures: automatically created sample data
– To fill testing database with predefined data before tests run
– database independent
– Require explicit loading with fixtures method within test class
• In directory 'test/fixtures':
– fixture stubs for each 'script/generate model'
• Formats
– YAML: good readability, file extension '.yml'
– CSV: comma-separated value file format, '.csv', easy reuse of
existing data in spreadsheet/database (save/export as CSV)
YAML-Fixtures
File 'persons.yml' of YAML-Fixtures in 'text/fixtures':
# This is a YAML comment!
david:
<< name1
id: 1
name: David Dwarf
birthday: 0010-01-01
profession: slingshoter
goliath:
<< name2
id: 2
name: Goliath Giant
birthday: 0000-02-22
profession: terrorizer
<< list of values
<< fixture-records separated by blank line
each fixture:
* 'fixture-name'
* list of colon-separated key/value pairs
CSV-Fixtures
File 'users.csv' of CSV-Fixtures in 'text/fixtures':
<< header: first line, commaid, username, password, intelligent, comments
separated list of fields
1, dhow, imstupid, false, I laugh ""Ha! Ho! Hu!""
2, admin, ihatedhows, true, What a mess!
<< list of value-records,
3, nobody, ilovetomock, true, "Nobody mocks you" one record per line
format:
4, nulpe,, false,
* each cell stripped of outward facing spaces
* comma as data: cell must be encased in
quotes
* quote as data: must escape it with 2nd quote
* no blank lines
* nulls achived by placing comma
CSV fixture names automatically generated: “model-name”-”counter”
users-1
users-2
...
Fixtures in Action
# allow this test to hook into the Rails framework
require File.dirname(__FILE__) + '/../test_helper'
# need to include a User for testing
require 'user'
class UserTest < Test::Unit::TestCase
fixtures :users
<< fixture-load method:
* destroys any data in users table
* loads fixture data into users table
def test_count_fixtures
* dumps the data into a variable for direct access
assert_equal 5, User.count
end
end
automatic load of the fixtures at the start of each test method
Hashes with Fixtures
Fixtures are basically Hash objects:
- direct access via generated local variable of the test case
Fixtures can get form of the original class:
- access to methods only available to that class
...
fixtures :users, :person
<< load multiple fixtures separated by commas
def test_user
users(:nobody) << returns Hash for fixture named david
users(:nobody).id << returns id-property of david
end
def test_person
david = users(:david).find
<< using find method to grab "real" david as Person
email( david.girlfriend.email, david.illegitimate_children )
<<now: access to
methods only available to a Person class
end
...
Mocking with Mocha
• Problem with Fixtures
– Fixtures makes testing slow (engage the actual DB)
– Fixtures allow invalid data
– Maintainability Challenges
– Fixtures are brittle
• That’s why we need Mocha
(http://mocha.rubyforge.org/)
– It provides stubs and mocks to simulate data,
especially data in the database
Installing Mocha
• sudo gem install mocha
• In rails, include in test/test_helper.rb
require ‘mocha’
Do Not Mock My Stub
• Stubbing (State Verification)
– Stubbing a method is all about replacing the method with
code that returns a specified result (or perhaps raises a
specified exception).
• Mocking (Behavior Verification)
– Mocking a method is all about asserting that a method has
been called (perhaps with particular parameters).
• it’s difficult (or impossible?) to do mocking without stubbing you need to return from the mocked method, so that the
code under test can complete execution
• http://www.martinfowler.com/articles/mocksArentStubs.html
Stubbing example (stub)
require 'test/unit'
require 'rubygems'
require 'mocha'
class TestProduct < Test::Unit::TestCase
def test_product
product = stub('ipod_product', :manufacturer => 'ipod', :price => 100)
assert_equal 'ipod', product.manufacturer
assert_equal 100, product.price
end
end
More stubbing example (stubs)
class View
attr :document
def initialize(document)
@document = document
end
def print()
if document.print
puts "Excellent!"
true
else
puts "Bummer."
false
end
end
end
require 'test/unit'
require 'rubygems'
require 'mocha'
class ViewTest < Test::Unit::TestCase
def test_should_return_false_for_failed_print
document = stub("my document")
document.stubs(:print).returns(true)
ui = View.new(document)
assert ui.print
end
end
Mocking example (expects)
class Enterprise
def initialize(dilithium)
@dilithium = dilithium
end
def go(warp_factor)
warp_factor.times
{ @dilithium.nuke(:anti_matter) }
end
end
require 'test/unit'
require 'rubygems'
require 'mocha'
class EnterpriseTest < Test::Unit::TestCase
def test_should_boldly_go
dilithium = mock()
dilithium.expects(:nuke).with(:anti_matter).at_least_once
enterprise = Enterprise.new(dilithium)
enterprise.go(2)
end
end
Expects More Mocking
• Methods in expects:
– at_least, at_least_once, at_most, at_most_once,
in_sequence, multiple_yields, never, once, raises,
returns, then, times, when, with, yields