Comments on: How not to implement a failing test

By: Arnis L.

Arnis L. — Thu, 22 Oct 2009 13:20:26 +0000

You should add those comments to your post. Now it sounds much more reasonable.

Btw – thanks for presentation. It was really nice.

By: bogardj

bogardj — Wed, 21 Oct 2009 15:49:26 +0000

@James

The point is to get to a failing assertion – not just a failing test. Assertion failures tell me if my code under test is working or not. Exceptions tell me nothing. It has nothing to do environmental constraints – it’s the critical “Red” step to prove my test is correct. Exceptions prove nothing.

By: James Gregory

James Gregory — Wed, 21 Oct 2009 13:42:39 +0000

I’m failing to see the distinction between a NotImplementedException and returning a default value. Both are simply a stop-gap mechanism for getting your code to compile so you can -see- the failure. In a less strict language you wouldn’t need this step, because your test would be immediately runnable, but without this luxury we’re stuck with either default values or exceptions. To favour one is personal preference, but decrying one over the other seems pedantic for a step that’s only necessary due to environmental constraints.

By: bogardj

bogardj — Wed, 21 Oct 2009 13:18:29 +0000

@James

The test failure provides no meaning. If a test fails because of the Execute step borking, my only recourse is debugging. Assertion failures provide the most documentation.

If your first goal in RGR is to fail a test, why bother writing the assertions yet? Seems like a waste of time if all you want to see is a failure, no matter where the source.

By: bogardj

bogardj — Wed, 21 Oct 2009 13:15:27 +0000

@Jaco

But now you’ve never shown that your “Assert” part of AAA works correctly. If “Red” is to do “Setup Execute” but never “Verify”, the “Green” part now has two points of failure.

By: bogardj

bogardj — Wed, 21 Oct 2009 13:14:10 +0000

@Brendan

That’s why you write a failing test first – you need to calibrate your test to make sure your test is correct.

By: James Gregory

James Gregory — Wed, 21 Oct 2009 09:38:53 +0000

Playing devils advocate. What’s not meaningful about throwing a NotImplementedException? It fails because it’s not implemented, certainly seems meaningful.

By: Jaco Pretorius

Jaco Pretorius — Wed, 21 Oct 2009 06:12:08 +0000

I don’t think I really agree with you – where is the rule that says the Red in RGR should be a meaningful failure? When I’m designing my tests I want to write the least amount of code possible to get the code compiling. That’s the first step – I just want to be able to run the test and see it fail. The next step is to write the simplest code to get the test to pass – this is usually something along the lines of ‘return 5;’

I suppose we are arguing semantics but my point is that this is such a short step in your design process that I would suggest that you do whatever you feel comfortable with.

By: Mike Murray

Mike Murray — Tue, 20 Oct 2009 22:22:12 +0000

I like how in Aaron Jensen’s MSpec, not implemented specs are considered neither passing nor failing; they are considered Not Implemented yet! The yellow of Red and Green testing.

By: Bill Sorensen

Bill Sorensen — Tue, 20 Oct 2009 17:31:36 +0000

@scott: In ReSharper 4.5, this is in Options, Languages, Generated members.