When Do You Specify Expected Parameters In Stub Methods?

I’m writing a spec with a mock object that mock object returns data to the class under test. In these situations, I don’t bother asserting that my mock object’s method was called because I know that if it’s not called the data I need in the class under test won’t be there and I’ll end up having other unexpected behavior. This falls under the general guideline of ‘test output and interactions, not input’.

In this specific situation, I am taking a value from user input and using that value to load up some data from a mock repository. I find myself wondering if I should specify the value that is being passed into the mock object’s method so that the mock will only return the data I need if the method is called with the right value. To illustrate in code, here are the two different ways I could do this.

1. Always return the data from the stubbed method call:

   1: private IAssetGroupRepository GetAssetGroupRepository()

   2: {

   3:     var repo = Mock<IAssetGroupRepository>();

   4:     AssetGroups = new List<Lookup>();

   5:     repo.Stub(r => r.GetGroupLookups(Arg<int>.Is.Anything)).Return(AssetGroups);

   6:     return repo;

   7: }

2. Only return the data from the stubbed method when the right argument is found:

   1: protected override IAssetGroupRepository GetAssetGroupRepository()

   2: {

   3:     var repo = Mock<IAssetGroupRepository>();

   4:     AssetGroups = new List<Lookup>();

   5:     repo.Stub(r => r.GetGroupLookups(1)).Return(AssetGroups);

   6:     return repo;

   7: }

The difference is on line 5 – the use of Is.Anything vs. a literal value of 1. It seems that the arguments should be specified when it matters what the arguments are… when the arguments are going to determine whether or not the right thing is being done. In this situation, it seems to me that the argument is important. If I’m not specifying the value that was selected when calling the GetGroupLookups, then my code has failed to account for the user’s input and it will likely produce the wrong behavior. The counterpoint to this is that the test where this stub definition lives becomes a little more brittle.

So the question is when should I return the data no matter what arguments are used vs. when should I only return the data when the right arguments are used? I know the answer is “it depends”, as that’s the only valid answer to any code question. :) But I’m looking for some input from the rest of the world on when they do / don’t require the right arguments and why.


Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Derick Bailey

Derick Bailey is an entrepreneur, problem solver (and creator? :P ), software developer, screecaster, writer, blogger, speaker and technology leader in central Texas (north of Austin). He runs SignalLeaf.com - the amazingly awesome podcast audio hosting service that everyone should be using, and WatchMeCode.net where he throws down the JavaScript gauntlets to get you up to speed. He has been a professional software developer since the late 90's, and has been writing code since the late 80's. Find me on twitter: @derickbailey, @mutedsolutions, @backbonejsclass Find me on the web: SignalLeaf, WatchMeCode, Kendo UI blog, MarionetteJS, My Github profile, On Google+.
This entry was posted in .NET, Behavior Driven Development, C#, Pragmatism, Principles and Patterns, Unit Testing. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.jamesthigpen.com James Thigpen

    I am having a giant moral problem with interaction tests such as these right now. I feel like they’re really testing implementation, but there’s no state a lot of times in these services to test.

    And I’m having similar questions such as this as to whether or not argument expectations are valuable.

    I guess I’m not any help, but I’m in the same boat as you I think.

  • mendicant

    If I don’t really care, for example if it’s coming from user input, I’d probably go with method 1.

    However, if I was using the AssetGroups that was returned in another stubbed call, then I would use method 2.

  • mendicant

    @James:

    I do the same thing. Mostly, I find myself hating it once its written, but when I’m writing it the first time I find myself enjoying such tests because it forces me to think out what I’m doing before actually writing the code.

  • http://shane.jscconsulting.ca Shane Courtrille

    I think that specifying parameters with stubs should be considered a test smell. If your intention is to test that an interaction is occurring correctly then you should write a test accordingly. The point of tests after all are to tell us when something is wrong. If a stub doesn’t get matched all we end up with is a bunch of failing tests and we have to pull out the debugger to figure out why. A mock expectation on the other hand will tell us exactly whats wrong.

    Of course that leads to the question.. how much mock testing should we really be doing? I’m still trying to figure that one out…

  • http://www.lostechies.com/members/derick.bailey/default.aspx derick.bailey

    @Shane,

    would you suggest #1 from the above code with an additional assetion in my tests, then, like this:

    repo.AssertWasCalled(r => r.GetGroupLookups(1));

    ?

    i’ve generally shy’d a way from that type of test for the reason stated at the very top of this post… though i do get your point about having to pull out the debugger to find where a problem is sometimes… i’ve had to do that on occasion – probably more often than I am remembering at the moment.

    hmm…. ponderings…

  • Mark Bourland

    Maybe I’m bogged down on the fact that you’re giving a sample of a repository call, but the only time I would care what parameter values are passed to a stubbed repository call is when I have one repository method that could return data that will be acted on differently based on what it is.

    For a stupid example: if my repository call returns a null value, I do X but if it returns a “good” value I do Y. In that case, I have to test when the same repository call might return null vs. any other value, so I would stub out something like:

    repo.Stub(r => r.GetGroupLookups(1)).Return(null);
    repo.Stub(r => r.GetGroupLookups(2)).Return(AssetGroups);

    Hopefully that makes sense.

  • http://www.lostechies.com/members/derick.bailey/default.aspx derick.bailey

    @Mark,

    your example is only half there – and i think my example may only be half there, now that I think about it…

    i think your example is dealing more directly with the repository as an input mechanism. in that case _how_ you are doing something based on the return value may be important in that situation. for example, if you are changing your interactions with a view interface, then you would write your assertions against the view interface interactions. in that case, i don’t think it’s important to specify the input to the repository call. the specific test should setup the repository stub to return the value that will cause the presenter to make the correct call to the view.

    i think my scenario is slight different because the repository call is both interaction based on input (from the user in this case) and its also input into the presenter. … it’s both input and interaction if that makes sense. so my stub in option #2 was kinda-sorta testing the interaction by requiring the right parameter… which makes me think @shane has the right idea and my response to him is something to look into.

    … still more ponderings …

  • http://nolanegly.edgewolf.com Nolan Egly

    Could be a bit more concrete with your example? What are you testing, the repository or the presenter?

    If you’re testing the presenter, you probably have a method invoked from the view, something like:
    public GroupData GetGroup(int groupId)
    {
    return Repository.GetGroupLookups(groupId);
    }

    In which case, I would verify GetGroup calls the repository with the proper group id. If you don’t, the test can pass for the wrong reason, and that’s a Bad Thing.

    For me the big discouragement in the past was a painful constraint syntax. Assert.WasCalled() makes life much easier, and I don’t think it makes the tests brittle.

  • http://melioratingmonkey.blogspot.com/ MM

    i would have one test that uses a mock like your response to shane indicates but i would make that the only thing that that test does.

    for actually using the fake repo, i would suggest not using the same helper method to get the repo for testing all conditions. i would always ignore the parameters and just configure the repo differently based on what i’m trying to test. so maybe you’d end up with a GetValidAssetGroupRepository method and a GetInvalidAssetGroupRepository method, or something like that.

    finally, when you’re actually testing the repository implementation(s), you can make sure that the parameters work correctly.

    on a semi-related note, my first ever blog post deals with state vs interaction testing

  • http://www.lostechies.com/members/derick.bailey/default.aspx derick.bailey

    @Nolan – correct on usage, and i went with the test having the id specified for the same reason.

    the possible brittleness of the test had less to do with the record/replay vs. AAA syntax and more to do with how easily the test will break if i need to make changes to the code. in this case, i think the confidence of calling the method with the right parameter is worth the price of a very small potential in breaking the test if i change the API of the repository call.

  • http://darrell.mozingo.net Darrell Mozingo

    @derick – I can see your point about specifying the parameter when you’re talking about simple argument types, but what about when you run into the same situation for a method that takes something a bit more complex (or two or three such complex arguments)? It gets hairy making sure the right parameters are passed in, i.e. having to check for matching id’s, equal instances, certain parameters, etc.

    I run into this mostly in my services, especially when it’s one fake talking to another fake – like getting something from a repository and passing it to an email service.

  • http://fernandozamorajimenez.blogspot.com/ Fernando Zamora

    @derick – I do not normally specify parameters unless the service call, in this case the repo, is called more than once (in the single method undert test) and I expect different results for each call.
    There has been cases in which I needed the repository to behave like the real thing. In those cases I hand roll a Fake/Double that mimics the real thing. That can arguably be just as brittle if the functionality of the real dependency changes… but in my case it got the job done and it proved some complex things that could otherwise only be tested in a multi-client host environment.

  • http://nolanegly.edgewolf.com Nolan Egly

    @Darrell,

    You might possibly use one “parameter object” rather than several parameters in the context your describing. The parameter object shouldn’t have behavior since it’s just shuttling data from service to another, ideally is immutable (all data set in ctor, no public setters), and implements an equivalence override of Equals().

    Then, you can create an instance of the parameter object in your test, and than verify that your fake email service received the equivalent (not necessarily the reference, because the intermediate object is constructing a new parameter object based on the repository results) parameter object.

    This makes parameter checking easier and less brittle, because there’s only parameter to verify. I’ve found this useful when testing presenters that are getting data from the repository and updating a view.

  • http://neilmosafi.blogspot.com Neil Mosafi

    Usually the first. When testing a particular method or behaviour I always first stub out the methods out to return some known response no matter what arguments are specified. This allows the method being tested to complete (e.g. without throwing a null reference exception because something was not returned from a stub) and once it has completed, I can perform the state based asserts that I need to.

    I will then look at the other methods being called and think “do I care about the arguments?”. If I do, I will write a separate observation for each argument I want to verify (e.g. x.AssertWasCalled). This makes the tests less brittle and It’s much easier to understand when something goes wrong.

    The only time I would specify arguments up front in the stub, is when I need to return a different response from the stubbed method depending on the argument.

    Hope that makes sense.
    Neil

  • http://shane.jscconsulting.ca Shane Courtrille

    Not exactly. While I would suggest going with option #1 as to if I would recommend adding the additional assertion test the answer is of course it depends…

    Testing interactions is one of those things that is very necessary but can also quickly lead to over specification which is a very stinky test smell. Personally this is one of those issues I have been struggling with for a very long time. I have definitely decided against asserting every single interaction. Now the question I keep having to ask is.. How important is this interaction? Is it worthy of a test?
    Well I guess that’s two questions :)

    Regards,

    Shane

  • http://www.truewill.net/myblog/index.php Bill Sorensen

    I’d specify the argument but use a constant in this case.