Making a Unit Test Framework agnostic Assertion in C#

 

At the beginning of the year I got some feedback that the assertions in the MvcContrib.TestHelper library were great and very useful but the problem with them is that we relyed on the NUnit assertions.  This was a problem for some users as they did not use nunit and did not want to include that dll in their codebase.  I also have been frustrated because NUnit is strongly named and since I am using other framework that add a little extra on top of nunit I have found that I know need to add all sorts of Assembly Redirect configuration in my app.config files in order to trick all  of the assemblies I am using to think they have the right version of NUnit.   It is quite a big mess when you really think about it.

So at the Alt.Net conference in Seattle I proposed a session to talk about Unit Test Framework agnostic assertions and helpers. I was quite lucky to have some very smart people in the room and on the kyte.tv feeds.  The end result that came out of the was this.  Just throw a custom exception and the test frameworks will pick it up.  I was told that this is how Rhino Mocks handles it’s assertions and that the test frameworks would eventually add my namespace to there code base so that the Call stack reported to the developer running his tests would not see the frames that were used internally by my assertions. Let me give you a visual so that it is a little easier to understand exactly what the problem is….

The first line in this example shows a line that is from my special assertion code.  I do not want that to show up when a user runs a unit test with my assertion.

 image

Here is my little assertion helper example.  Since I just created a standard exception I got the default behavior.

image

 

The way to fix this is to throw a special exception which knows how to remove specific frames out of the call stack.  This code was adapted from the xUnit source code and is working quite well in the MvcContrib Test Helper Library

image

With some simple string manipulation of removing lines that contain the namespace of my exception it is really easy to reuse this exception in an Unit Test framework Agnostic helper library.

 

Here is the result of an assertion that failed using this assertion.  As you can see the test runner does not show lines that my assertion framework contain the call stack stops at the last line in my unit test code.  Which is exactly what we want.  That way there is no question as to where the developer using this needs to look to get information about the test failure.

image

This code is located in the MvcContrib TestHelper library located at http://MvcContrib.org in case you want to dig deeper into the code.  I hope this helps others as it really took some time to get to this approach.. It seemed totally obvious to the developers of the unit test frameworks but for us mere mortals it took a little extra digging to really understand how the Unit Test runners work. 

 

Comments .. Questions ???

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Eric Hexter

I am the CTO for QuarterSpot. I (co)Founded MvcContrib, Should, Solution Factory, and Pstrami open source projects. I have co-authored MVC 2 in Action, MVC3 in Action, and MVC 4 in Action. I co-founded online events like mvcConf, aspConf, and Community for MVC. I am also a Microsoft MVP in ASP.Net.
This entry was posted in .Net, altnetseattle, c#, mvccontrib, testing, Tools, Unittests. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://lunaverse.wordpress.com Tim Scott

    This is a cool way to shed the superfluous stack items. But I’m not sure I’d be willing to give up the nice messages that the test framework’s assertions give me. There’s a lot of value in seeing something like this:

    String lengths are both 3. Strings differ at index 0.
    Expected: “foo”
    But was: “bar”
    ———–^

    I recently created something that aims at addressing the problem of portability across testing frameworks, although it does not remove the extra stack frames.

    http://code.google.com/p/shouldit/

  • http://www.lostechies.com/members/erichexter/default.aspx erichexter

    @Tim this is only the solution for removing the call stack frames. I would like to see something like NBehaves assertion syntax be implemented using this method. Instead of the variations for each framework they could just implement one assertion library that would work with all the test runners and frameworks.