Putting mocks in their place

Awhile back, I talked about 3 simple rules for Rhino Mocks:

  1. Use the static MockRepository.GenerateMock method.  Don’t use an instance of a MockRepository, and don’t use the GenerateStub method.
  2. If the mocked instance method returns a value, use the Stub method to set up a result for that value.  This is an indirect input.
  3. If the mocked instance method does not return a value (void), use the AssertWasCalled/AssertWasNotCalled methods.  This is an indirect output.

This can be simplified even further – if we’re following command-query separation, then it really comes down to having two types of methods:

  • Commands – these are Action-based delegates.  They perform an action, but return nothing (void)
  • Queries – these are Func-based delegates.  They answer a question, but they do NOT modify state

If we follow this rule, then our Rhino Mocks rules become:

  • Use Stub for Queries
  • Use AssertWasCalled for Commands

After using flavors of TDD, BDD and so on for a few years, I’ve had a bit of an epiphany.  Mocks, stubs, fakes and the like are most valuable when they replace external dependencies.

Why?

Unit tests with mocks and stubs leak implementation details into the test, leading to a test that’s coupled to the internal implementation of how the tested class does its job.  Tests coupled to the implementation of the class under test are a barrier to refactoring.  If our tests become a barrier to refactoring, it kills one of the purported benefits of refactoring – providing a safety net for refactoring!

So where do mocks belong?

Where things are hard to test, like databases, web services, configuration files, and so on.  Components outside the core logic of our application, that we don’t have control over.

An example

Back in the original post, we looked at an order processor:

public class OrderProcessor
{
    private readonly ISmtpClient _client;
    private readonly IOrderSpec _spec;

    public OrderProcessor(ISmtpClient client, IOrderSpec spec)
    {
        _client = client;
        _spec = spec;
    }

    public void PlaceOrder(Order order)
    {
        if (_spec.IsMatch(order))
        {
            string body = "Huge frickin' order alert!!!rnOrder #:" + order.OrderNumber;

            MailMessage message =
                new MailMessage("sender@email.com", "salesdude@email.com",
                                "Order processed", body);

            _client.Send(message);
        }

        order.Status = OrderStatus.Submitted;
    }
}

Suppose that we don’t like how we chose how the OrderProcessor matched on the specification, and decided to go a completely different direction.  Yes, we can’t test email, but let’s look at the implementation details in our test:

[Test]
public void Should_send_an_email_when_the_order_spec_matches()
{
    // Arrange
    var client = MockRepository.GenerateMock<ISmtpClient>();
    var spec = MockRepository.GenerateMock<IOrderSpec>();
    var order = new Order {Status = OrderStatus.New, Total = 500m};

    spec.Stub(x => x.IsMatch(order)).Return(true);

    var orderProcessor = new OrderProcessor(client, spec);

    // Act
    orderProcessor.PlaceOrder(order);

    // Assert
    client.AssertWasCalled(x => x.Send(null), opt => opt.IgnoreArguments());
    order.Status.ShouldEqual(OrderStatus.Submitted);
}

The whole need for an OrderProcessor to need an IOrderSpec, or that the IsMatch method is called is an implementation detail.  If we want to make large, interesting changes to this class, we’ll need to just ditch these tests altogether.  And if we have to delete tests every time an implementation changes, how is that a safety net?  It’s not.

Instead, I’ll write my tests quite a bit differently, ones where dependencies don’t show up unless they truly cannot be used, and even then, I’ll try and build out fake implementations that are easily re-usable.  In the next post, I’ll go through this style of test that allows refactoring much more easily than the implementation leaks of the above test.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in TDD. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://blog.jorritsalverda.nl Jorrit Salverda

    I run into exactly the same problems with refactoring, that I have to ditch or update a lot of tests!

    For mocking stuff I use the excellent AutoMocker paired with StructureMap. Wat would be cool is if we could tell AutoMocker to use all concrete implementations except for a defined set, namely the external dependencies. Do you know if there’s an easy way to do that? Looking forward to the next post…

  • Rob Miles

    Hey Jimmy,

    Good post – totally agree with your guidelines for using RhinoMocks.

    The times when I find that mocks are useful apart from for external dependencies though are when testing a class that is essentially orchestrating a complex business process by delegating to many other classes. In this case I find it useful to mock the classes that are being delegated to, as this allows simple tests against the orchestrating class that check that all the high level actions are being performed. Without this then you can end up with the tests for the orchestrating class being extremely complex. How do you deal with this sort of scenario?

    One other question – are you aware of any differences between GenerateMock and GenerateStub other than that GenerateStub registers PropertyBehaviour against all properties on the stubbed class?

    Cheers,
    Rob

  • http://blog.robbowley.net Rob Bowley

    Jimmy, I’m afraid your epiphany is based on a well worn misconception about mocking and the need which resulted in the creation of the concept. The anti-patterns you and others who have commented are discussing are well known when mocking is used improperly.

    Nat Pryce and Steve Freeman (who invented JMock and as good as invented mocking) find they have to spend a lot of their time fighting against these misconceptions.

    May I suggest the following paper written by Nat Pryce et al (in 2004) explaining the process of using Mocks and also some widely held misconceptions (your one is even mentioned in the abstract):

    http://static.mockobjects.com/files/mockrolesnotobjects.pdf

    I would also suggest reading their book:

    http://www.amazon.com/Growing-Object-Oriented-Software-Guided-Tests/dp/0321503627/ref=sr_1_1?ie=UTF8&s=books&qid=1294392916&sr=8-1

    …and checking out the GOOS google forum:

    http://groups.google.com/group/growing-object-oriented-software

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Rob Browley

    Yes, I’ve been referred to the GOOS book by a few folks now, but it’s just down on the reading queue.

    One thing I’ve seen is that tests that specify interactions with immediate neighbors have increasingly little value for me. The only time I’ll really pull out mocks these days are at the system boundary, or when my classes have interesting behavior that I’d like to isolate. Otherwise, meh.

  • http://www.lostechies.com/blogs/thatotherguy/default.aspx Gregory

    So, are you saying ‘never or almost never specify the interactions of objects and their immediate neighbors’. I would consider this a ‘only write system tests’ approach – where system is defined as a set of related objects that interact to meet a higher level expectation. Is that close to what you mean?

    You mention in your last comment you would specify interactions ” . . . when my classes have interesting behavior that I’d like to isolate”. Could you be a little more specific (or is that coming in the next post)?

  • http://simpleprogrammer.com John Sonmez

    Keep banging that drum, you are headed the right direction I believe. We overuse mocking and complicate unit tests, mocking only external dependencies seems like the right approach to me.

  • http://www.ntcoding.co.uk ntcoding

    Please could you show how you would do a typical “create” scenario in say a web app. Let’s say you are creating a “fish” object and saving it to the repository.

    Would you mock the repository and assert methods were called? Would you mock only the “data context” for the repository?

    Interesting blog post on a topic I have been debating (not arguing about :) ) with a colleague the last few days.

    If you can’t give me an example, then I can only look forward to the next episode.

    Thanks

  • http://isaiahperumalla.wordpress.com isaiah

    Hi Jimmy
    “Tests coupled to the implementation of the class under test are a barrier to refactoring” if that is the case, the tests are hinting there is a issue with the design, as Rob suggested the GOOS book shows there is another school of thought where mocks are used an aid in designing OO code. one of the key things i got from GOOS book and chatting with steve and nat was distinguishing between internals and peer objects.

  • Josh

    External dependencies should not be mocked. Any code you do not own or have control over should not be mocked. Where such code is used, integration tests should be used. This is the only way to ensure the uncontrollable code is in fact doing what you expect, and that you code is interacting correctly with it. And you won’t know this if you mock it out.

    If you need to separate out this uncontrollable code, you need to use an adapter layer. Integration test the adapter and uncontrollable code by mocking your calling code. You can now unit test your calling code by mocking the adapter.

    This info was from GOOS. I really recommend this book.

  • http://www.lostechies.com/members/derekgreer/default.aspx derekgreer

    It sounds like you’re advocating the use of XP “Programmer Tests” over isolation tests (what most call unit tests). I can see pros and cons with both approaches. Programmer tests are less brittle because they don’t make as much use of test doubles, but they also don’t provide precise feedback about why a test failed (a single regression can result in many tests failing) or force decoupling through isolation. Isolation tests tend to force the designer to program to interfaces, but can be less resilient to change as compared to broader-brush tests.

    I currently favor higher level acceptance tests which drive the creation of lower lever isolation tests because I like the specific feedback and the discipline it seems to provide (especially for teams new to TDD/BDD), but I’m looking forward to reading the arguments you present in your follow-up post.

  • http://www.ntcoding.co.uk ntcoding

    @derekgreer

    That comment you make helps to put things into perspective for me. If I’m right then your essentially saying you can’t have unit tests (that isolate the smallest unit of code) without mocks.

    Going back to my comment above, if you mock the repository then you do have a fine-grained unit test. If you don’t mock the repository you have the “XP Programmer Tests” which can cause controller tests to fail if the repository tests fail.

    I think we are all looking forward to the next installment. You have built a lot of expectancy now Mr Bogard.

  • http://www.lostechies.com/members/derekgreer/default.aspx derekgreer

    @ntcoding

    It isn’t that programmer tests never use test doubles, it’s just that there isn’t an emphasis on isolation and therefore real collaborators are used when possible. Collaborators which encapsulate access across a process boundary (such as a repository) would generally be stubbed within a programmer test as well. Perhaps a better example for contrast might be how a class with a calculator dependency might be tested with a unit test verses a programmer test.

  • http://www.martinsharp.wordpress.com - Martin

    I can see wher eyoru coming form but i think you need balance in your assertions about mocking especially in a BDD context i would offer this post as a balancing perspective if i may.

    http://martinsharp.wordpress.com/2011/01/09/interface-first-development-and-mocking-overview/

    if you would like to discuss please email me on rebeccamartin@me.com

  • Uttam Kini

    I agree that mocks are misused. But I think if used correctly, they can breed good OO and flexible code.
    Even for code without external dependencies, if I have a sufficiently complex piece of business logic, then without mocks my unit test becomes a huge suite of integration tests.
    Having said that, if I see a unit tests with a lot mock assertWasCalleds then it is a code smell for me, which indicates the class under test is probably doing too much.

  • http://www.facebook.com/vasilio Vasilio Ruzanni

    How about specific implementation when our Command invoker (CommandBus) is returning at least Ack/Nak kind of result (on CommandBus.Broadcast(command) call) so we do always know that Command is being processed?
    Probably, we don’t need an AssertWasCalled at all?