I Want Assert.NotYetImplemented(); In .NET Test Frameworks

One of my coworkers recently tweeted this:

Just wrote a bunch of failing unit tests that describe how a class should work. Leaving them as failing, so I know where to start tomorrow.

I do that a lot with c# and nunit. It helps me to figure out where I need to go, what tests are going to be organized where, etc. I certainly don’t expect my first pass at the tests to be the final layout, and I don’t try and get every last test laid out all at once. I just lay down the ones that I know of immediately. I also tend to add new test shells as I am working in the tests and implementation, as I think of scenarios. This gives me a good place to start and a overview of what the code should be doing at a point right in between the user story and the implementation. Leaving the tests failing (with a “throw NotImplementedException()” or an “Assert.Fail(“not yet implemented”)”) gives me the freedom to go about my business and focus in on what i should be doing next, without having to worry about keeping track of the things i will be doing in the future. It’s quite nice, really. I’ve been doing this for a while now, and it works for me.

But… here’s the thing that started bothering me this morning: if I were writing ruby instead of c#, there’s no way I would leave failing tests lying around like this. Worse yet, if I were writing code in collaboration with any of my coworkers, I would not want to burden them with these failing tests. After all, if I commit these test to source control, then I’ll fail the build and that’s definitely not what I want to do. So, instead, I find myself keeping my code locally (in git, of course… local source control safety) and not sharing with anyone else until I’m done with the tests that I laid out.

This is counter to the collaborative, incremental nature of software development that we should be employing. The only time I should horde my code on my local box is when I am in the middle of working on something and it is obviously broken. Otherwise, I should be pushing my changes into the shared source control whenever I have parts of it working.

 

Taking A Lesson From RSpec

If I were writing ruby and rspec tests, I wouldn’t have this problem. I could simply omit the code block for my tests like this:

   1: describe "when something is happening here" do

   2:   it "should do this thing"

   3:   

   4:   it "should do that thing"

   5:   

   6:   it "should do some other thing"

   7: end

by leaving out the “do … end” code blocks in each of the “it” tests, rspec will gracefully produce the type of results that allow me to have my not yet implemented tests, but still share my code and my tests with the rest of the team:

image

The tests come back as a nice yellow color with a message stating that they are not yet implemented. It is easy for me to see where I should be working next, yet still commit this code to the shared source control and have other team members and the build server run without being clobbered by me failing tests.

 

I Want Assert.NotYetImplemented

That’s what I want… I want the ability to have a not yet implemented test, like rspec. It should not be too terribly difficult for the Nunit, xUnit, MBUnit, MSpec, and other test framework crews to add in this code. Something as simple as this would work:

   1: [Test]

   2: public void some_test_that_i_will_care_about_soon()

   3: {

   4:   Assert.NotYetImplemented();

   5: }

And then we can get the CI build servers like CCNet, Hudson, TeamCity, etc, and the test runners like Resharper, Gallio and TestDriven.NET to join in and update how they report tests. It will take time, of course… there are a lot of players in the .NET community that would need to coordinate this type of change… but it’s worth it. It gives us a lot more flexibility in how we operate in our teams.


Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Derick Bailey

Derick Bailey is an entrepreneur, problem solver (and creator? :P ), software developer, screecaster, writer, blogger, speaker and technology leader in central Texas (north of Austin). He runs SignalLeaf.com - the amazingly awesome podcast audio hosting service that everyone should be using, and WatchMeCode.net where he throws down the JavaScript gauntlets to get you up to speed. He has been a professional software developer since the late 90's, and has been writing code since the late 80's. Find me on twitter: @derickbailey, @mutedsolutions, @backbonejsclass Find me on the web: SignalLeaf, WatchMeCode, Kendo UI blog, MarionetteJS, My Github profile, On Google+.
This entry was posted in .NET, Agile, AntiPatterns, C#, Community, Continuous Integration, Resharper, RSpec, Ruby, Test Automation, Tools and Vendors, Unit Testing. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.lostechies.com/members/jagregory/default.aspx James Gregory

    MSpec already does that, just omit the body from a should.

    • http://www.Marisic.Net dotnetchris

       I had some confusion on what exactly this meant, mainly from omitting the body specifically . What James was saying is that you should omit the entire assignment of an It assertion.

      It should_have_name_foo;

      Will show up as inconclusive on test runners.

  • http://aspiringcraftsman.com Derek Greer

    It isn’t the language, it’s your framework. Quit fighting it and just move to MSpec already :)

  • http://www.lostechies.com/members/jflanagan/default.aspx Joshua Flanagan

    You can achieve the rspec behavior by using the Ignore attribute on the test, or by calling Assert.Ignore().

  • http://www.coryfoy.com Cory Foy

    You could just use the Ignore attribute.

    http://nunit.org/index.php?p=ignore&r=2.2.8

    [Test]
    [Ignore("Not Yet Implemented")]
    public void some_test_that_i_will_care_about_soon()
    {
    //Whatever
    }

    Even Better, create a custom attribute which inherits from Ignore called – yep – NotYetImplemented.

    Cory

  • Akash Chopra

    How about Assert.Ignore()? I think this is available in most testing frameworks.

  • http://www.si-w.co.uk/blog Simon

    If you wanted failing tests to remind you where you left off, you could use Assert.Fail(“Not yet implemented”) in the test body.

  • http://www.codebork.com/ Alastair Smith

    Have you considered Assert.Inconclusive(“Not yet implemented”)? This should give you the nice yellow test result you seek, and should work with existing CI software too. It’s definitely in NUnit 2.5, maybe 2.4 also.

    IgnoreAttribute is also a good option, as other commenters have mentioned. There may be important semantic differences between Assert.Inconclusive() and IgnoreAttribute that I’ve missed.

  • Harry Steinhilber

    MSpec already has this functionality. Simply declare but don’t assign the It delegate:

    It should_do_something_I_will_care_about_soon;

    This will generate a not implemented message very similar to how RSpec does it.

  • Dan

    Could you create an Assert extention method that just called Assert.Ignore() ?

  • Dale

    There is also the option of making the class or method under test throw a NotImplementedException and expect that exception in your test.

  • http://trycatchfail.com/blog Matt

    I second (third?) the call for the [Ignore] attribute. I committed a few “stub” tests just last night using that approach. :)

  • http://www.lostechies.com/members/derick.bailey/default.aspx derick.bailey

    to the peeps that suggested Assert.Fail or throwing a NotImplementedException: you might want to read the blog post you are commenting on. :)

    to everyone else: good suggestions! I’m not a fan of Ignore, honestly. i think the semantics between ignore and not yet implemented are significant, but that would at least work. i’ll have to try out the Assert.Inconclusive, too. I was not aware of that assertion.

  • http://igorbrejc.net Igor Brejc
  • http://fooberry.com Mark

    We just renamed out our FactAttribute to ThenAttribute and then subclassed that to EventuallyThenAttribute and added a message for all skipped tests.

    [Then] public void the_something_happens(){}

    [EventuallyThen] public void the_something_happens(){}

    We’re dragging out feet on moving to MSpec as well.