Transcript PPT - Scandinavian Developer Conference

Capture-Replay Mocks
Scandinavian Developer Conference
4th April 2011
Geoff Bache
Overview
•
•
•
•
•
What is a “mock”?
What do “coded mocks” look like?
Introducing Capture-Replay mocks
CaptureMock, with examples
Demo
Copyright © 2008 Jeppesen Sanderson, Inc.
Jeppesen Proprietary and Confidential
Slide #
What are “mocks”?
• “Simulated objects that mimic the behaviour of real objects in
controlled ways” (Wikipedia)
• More strictly:
• “mocks” actively expect and assert behaviour
• “stubs” passively return hardcoded values
• “mocks and stubs” are collectively called “fakes” or “test
doubles”.
• Terminology unfortunately not used consistently.
Copyright © 2008 Jeppesen Sanderson, Inc.
Examples of when to use mocks/stubs
• When using the real code in your test would be
 Hard to analyse (e.g. images created)
 Too slow (e.g. databases)
 Indeterministic (e.g. current time)
 Hard/expensive to set up (e.g. network errors)
 Impossible (because it doesn’t exist yet)
• As a design strategy (Behaviour-Driven Development)
Copyright © 2008 Jeppesen Sanderson, Inc.
Mocking in Java (Mockito)
//You can mock concrete classes, not only interfaces
LinkedList mockedList = mock(LinkedList.class);
//stubbing
stub(mockedList.get(0)).toReturn("first");
stub(mockedList.get(1)).toThrow(new RuntimeException());
//following prints "first"
System.out.println(mockedList.get(0));
//following throws runtime exception
System.out.println(mockedList.get(1));
//following prints "null" because get(999) was not stubbed
System.out.println(mockedList.get(999));
//Checks that we called “get(0)” at some point
verify(mockedList).get(0);
Copyright © 2008 Jeppesen Sanderson, Inc.
Mocking in Python (Mock)
# Fix method return values and verify they were called
from mock import Mock
my_mock = Mock()
my_mock.some_method.return_value = "value"
assertEqual("value", my_mock.some_method())
my_mock.some_method.assert_called_once_with()
# Set up expectation we will raise exceptions...
my_mock.other_method.side_effect = SomeException("message")
assertRaises(SomeException, my_mock.other_method())
Copyright © 2008 Jeppesen Sanderson, Inc.
Stubbing in Python (“monkey patching”)
# Example of ”monkey-patching”
import time
def hardcoded_asctime():
return 'Thu Mar 24 16:12:44 2011'
# We monkey-patch the standard library for our test
time.asctime = hardcoded_asctime
class ProductionClass:
def method(self):
print ”The time is”, time.asctime()
real = ProductionClass()
# Will print our faked, deterministic answer
real.method()
Copyright © 2008 Jeppesen Sanderson, Inc.
Problems with using mocks
• Involves writing sometimes complex code
 Takes time to write
 Has to be maintained
• Easy to create a lot of dependency on implementation details
• If the “real code” changes its behaviour :
 The tests stay green
 But the system doesn’t work any more...
Copyright © 2008 Jeppesen Sanderson, Inc.
Capture-Replay Mocks
• Generate mocks by running the real code and capturing how it
behaves
• Results in a normal test that can be run in two ways
 using the stored mocks
 or regenerating them from the real code
• No mock-code to write, just say what you want to capture
• Much easier to keep in synch with real code
Copyright © 2008 Jeppesen Sanderson, Inc.
It's not “capture-replay” like EasyMock!
# Some mock frameworks just use “capture-replay” as a syntax
import mocker
# We start in the “capture phase”...
my_mock = mocker.mock()
my_mock.some_method()
mocker.result("value")
# and then we “switch to replay” when we're ready to test
mocker.replay()
#... test code
Which is basically just a more verbose way to express:
import mock
my_mock = mock.Mock()
my_mock.some_method.return_value = "value"
#... test code
Copyright © 2008 Jeppesen Sanderson, Inc.
CaptureMock recording
CaptureMock
Record
traffic
System
under
test
Copyright © 2008 Jeppesen Sanderson, Inc.
3rd party
module
CaptureMock replay
System
under
test
Copyright © 2008 Jeppesen Sanderson, Inc.
CaptureMock
3rd party
module
CaptureMock : doing it in practice
• CaptureMock is a tool written in Python that can do this with
 Python code (modules and attributes)
 Command-line calls (subprocesses)
 Plain-text messaging (over sockets)
• “Python code” part relies on Python’s dynamic features
• Other parts are language-independent
• Plain-text messaging assumes a synchronous model
• each socket sends one message and gets one reply.
Copyright © 2008 Jeppesen Sanderson, Inc.
A Python CaptureMock test
# test_email.py
from capturemock import capturemock
class ProductionClass:
def method(self):
# call some code resulting in email being sent
self.something()
@capturemock(”smtplib”)
def test_email_sending():
real = ProductionClass()
real.method()
I can then run:
$ env CAPTUREMOCK_MODE=1 test_email.py
which will actually send the email
and record the interaction with smtplib to a file called ”email_sending.mock”
Copyright © 2008 Jeppesen Sanderson, Inc.
A CaptureMock recorded mock (Python)
<-PYT:import smtplib
<-PYT:smtplib.SMTP()
->RET:Instance('SMTP', 'smtp1')
<-PYT:smtp1.connect('machine.site.com')
->RET:(220, 'machine.site.com ESMTP Sendmail; Tue, 9 Feb
2010 14:32:54 +0100')
<-PYT:smtp1.sendmail('me@localhost', ['tom', 'dick',
'harry'], '''From: me@localhost
To: tom,dick,harry
Subject: Hi Guys!
I love you all!
''')
->RET:{}
<-PYT:smtp1.quit()
When I run without CAPTUREMOCK_MODE, no email will be sent
and if the calls received differ from above, an exception will be thrown.
Copyright © 2008 Jeppesen Sanderson, Inc.
A command-line CaptureMock test
# update_and_build.sh
# We update some code from CVS and then try to build it
returned=`cvs update –dP /path/to/my/checkout`
# logic using stuff written on stdout etc
...
make
Here I put the following in .capturemockrc to tell it to capture calls to CVS
[command line]
intercepts = cvs
and then run
$ capturemock --record cvs_calls.mock update_and_build.sh
which will perform the update for real and record to ”cvs_calls.mock”
Copyright © 2008 Jeppesen Sanderson, Inc.
A recorded mock from a subprocess
<-CMD:cvs update -dP /path/to/my/checkout
->FIL:checkout
->OUT:U subdir/myfile.txt
->ERR:cvs update: Updating .
cvs update: Updating subdir
Note the ”FIL” line refers to stored copies of the files changed by cvs.
They are stored in a directory called ”cvs_calls.files”.
To replay this, we run
$ capturemock --replay cvs_calls.mock update_and_build.sh
maybe on a machine where CVS isn't installed.
CaptureMock will pretend to be CVS, reproducing the
standard output, standard error, exit code and files written.
Copyright © 2008 Jeppesen Sanderson, Inc.
A client-server CaptureMock test
Assuming I have a client that sends strings to a server,
which replies with how long they are,
and a script that runs both together:
$ capturemock --record communication.mock run_client_server.sh
Something like this will appear in communication.mock:
<-CLI:Here is a string
->SRV:Length was 16
<-CLI:Here is a longer string
->SRV:Length was 23
I can then use this data to test either the client or the server
in isolation from the other.
Copyright © 2008 Jeppesen Sanderson, Inc.
Demo : graphs and matplotlib
Copyright © 2008 Jeppesen Sanderson, Inc.
Examples of when to use CaptureMock
• When using the real code in your test would be
 Hard to analyse - yes
 Too slow - yes
 Indeterministic - yes, with care (record mode may fail)
 Hard to set up - possibly (record mode still hard to set up)
 Impossible because it doesn’t exist yet - no
• As a design strategy (Behaviour-Driven Development) - no
Copyright © 2008 Jeppesen Sanderson, Inc.
Conclusions
• Existing mocking approaches
– are labour-intensive when non-trivial code is mocked out
– don't provide an easy means to verify the mock against real code
• CaptureMock
– provides a means to (re-)generate mock information from the code
– and two "modes" to select from each time tests are run
Copyright © 2008 Jeppesen Sanderson, Inc.