The first 2 hours of TDD are the most painful

Quick Background

I admit, I’ve had a lot of failures with TDD. Mostly they were solo projects and I didn’t have the discipline and self control enough to stick with it. I ALWAYS regretted not sticking with it. However poorly I was doing it, it always gave SOME value and saved me from several bugs (earning its keep and paying back with dividends all the time I spent on it).  And the successes with TDD I have had have been rocky, at best.  I usually felt ashamed and embarrassed by this, seeing how there’s all these noted luminaries and successful practitioners on my Blog Roll talking about optimizing, tweaking and even redefining the style of TDD as BDD.

In my selfish desire to find out their secret, I worked with Ray Houston to try to trick a bunch of really good software developers (many of whom already had lots of TDD experience) to see if I could get them to teach me their secret ways so that I might become more proficient and successful at TDD. I didn’t expect it to work very well. BOY WAS I WRONG.

I was kind of down on the CodingDojo in my last post on the subject. I was bummed because some people came from great distances and left with less than I had hoped they would. I was bummed that I didn’t serve them better.  But, there were some really great things that came out of it. It did spurn a bunch of people to get more serious about what they were doing and take a second look at how they were doing it. But more importantly, it re-instilled confidence in people (including me) who had been wavering and unsure of where to go or how to do TDD more effectively.

It was truly great to see people whom I thought had TDD down cold, experience some of the same problems I have been having and not have a quick answer for it.  This reminded me the most important thing about TDD…

TDD is not a solved problem, yet.

You cannot minor in TDD at college. Heck, you cannot take a graduate level course on TDD at college.  TDD is not a fully solved problem. It’s a practice which has been proven, through experience of many, to be a useful, worthwhile, and valuable tactic for achieving higher quality, more agility (through change confidence), and measured, disciplined design.

Armed with this reassurance that I wasn’t totally a heel, I really started to get into the discussions, the debates, and, above all, the code at the Dojo.  I started jotting down some of the questions that were being raised and not answered all that great in the hopes that I could spend some time thinking about them.

I plan on writing more about these things as the weeks/months go on (especially when I start on with Jeremy and get to practice them more thoroughly and have more substance to write about).  But in the mean time, I thought I’d do a thought dump and see if anyone had any good rules of thumb, concrete advice, or even ‘laws’ to go by. I’ll be updating this post with any interesting comments that appear so that you don’t have to go hunting through them to find any gems (of course, all the comments are gems — except for Anonymous Blog Coward ones — but some are worth bringing to the top).

(P.S.- I’m trying to keep the BDD style questions out of here, that will probably be another post)

TDD Practice Questions

  • How much up-front design do you tolerate? How do you know when to stop (i.e. ‘when algorithms start getting discussed, it’s testing time’)?
  • What about certain “I just know we’ll need this” type stuff (i.e. putting a try/catch{Console.WriteLine(ex);} in your console main() method?)
  • When writing a test, in order to even get it to compile, you have to build an interface or two, a concrete class with some NotImplementedExceptions in it, etc. How far do you allow that to go?
  • If, during the middle of testing on a story, you realize that your up-front design wasn’t correct, do you stop and discuss with your pair right then, do what you need to do and proceed, or do you pull back and go back to full pre-test design mode on that story?
  • When proceeding to a new story, you discover that a test you wrote for a previous story no longer reflects the requirements. Do you refactor that test immediately, mark it as ignored until you finish the current story and cycle back to the ignored one? Something else?
  • If the new story’s requirements involve a slight tweaking to an existing test, do you tweak it, or make a new one and discard the old one (i.e. ‘No changing existing tests!’ or ‘Only change if it’s a minor compiler issue, but otherwise don’t change it’)?
  • If you’re, say, building up your model and it passes tests, but you’re seeing that it’s infantile and that the next few upcoming stories will produce significant changes to produce a new emergent model, is it appropriate to step back and consider a larger refactoring, or do you keep plugging and make the changes into the existing model even if it could benefit from some housekeeping that is otherwise out of scope?

TDD Style Questions

  • Assert.That(x, Is.EqualTo(y)) or Assert.AreEqual(y,x)?
  • How many asserts/test? Any caveats?
  • How do you know what to put in the SetUp method?

TDD Mechanics Questions

  • Is a refactoring tool absolutely necessary?
  • If so, what is the bare minimum features that the tool would need to facilitate decent TDD?
  • Integrated IDE test runner or a background source-watching auto-runner?

That’s most of what I wrote down with a few extra thrown in for good.  I’d love to hear some of your thoughts on these questions.

About Chad Myers

Chad Myers is the Director of Development for Dovetail Software, in Austin, TX, where he leads a premiere software team building complex enterprise software products. Chad is a .NET software developer specializing in enterprise software designs and architectures. He has over 12 years of software development experience and a proven track record of Agile, test-driven project leadership using both Microsoft and open source tools. He is a community leader who speaks at the Austin .NET User's Group, the ADNUG Code Camp, and participates in various development communities and open source projects.
This entry was posted in CodingDojo, Programming, TDD. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • jdn

    Questions like:

    Assert.That(x, Is.EqualTo(y)) or Assert.AreEqual(y,x)?

    almost guarantee lack of progress and should be banned. It doesn’t matter.

  • So I guess I shouldn’t ask your opinion on x++ or x+=1, huh? ;)

  • Jason

    Assert.AreEqual(y,x) is fewer key strokes

    How many asserts/test?
    in theory each test should assert exactly 1 function. whether your testing the state of an object, or the behavior of the method.
    example: if your testing a DTO has the properties set correctly, then each property should be it’s own test. or if the results of the method under test returns different values/throws exceptions each outcome should be it’s own test. Following the SRP this comes natural as each object/function only does 1 thing. if you find yourself with many assertions in 1 test this is a sure sign to refactor.

    How do you know what to put in the SetUp method?
    i use the setup method to mock dependencies like this. I use Rhino.Mocks for my mocking framework.
    public void setup()
    mockery = new MockRepository();
    mockTask = mockery.DyanmicMock();
    public void This_is_my_test()
    IResult mockResult = mockery.DynamicMock();
    using (mockery.Record())
    Expect.Call(mockTask.SomeFunction()).Return(IResult );
    using (mockery.Playback())
    Assert.AreEqual(mockResult, CreateSUT.DoSomething());

    private IObjectUnderTest CreateSUT()
    return new ObjectUnderTest(mockTask);
    I have read a few posts out there about automocking containers that people use on large enterprise systems. I think is build off of StructureMap. I 1st heard of it from JP Boodhoo He picked it up from another developer.

    Is a refactoring tool absolutely necessary?
    You don’t know what your missing, until you see what your missing. I started using ReSharper this fall and it makes a world of difference. before than I thought how much time would it really save.

    Integrated IDE test runner or a background source-watching auto-runner?
    the common setup I see is VS, ReSharper, Nant, [x]Unit, CC.Net. I’m using MbUnit. this has an autorunner feature, which works great on small projects, but is really slow once the project grows(because of debugging, not the framework itself). At this point I switch to GUI or Console runner.
    I haven’t started using Nant yet, but it’s on my list of 2008. I haven’t touched CC.Net yet either.

    from my perspective the developers i follow only use VS to write code. any buliding, testing, profiling, versioning is done through other tools. Nant seems to be at the core of all this.

    for me the biggest change from manual testing to TDD/testing was trusting my code, more than my eyes.

  • See my blog post in response to all of your questions. Hopefully it will be useful information. I’ve asked all the same questions in the last year, so I think I have some good insight into this.

  • Regarding the need for a refactoring tool: I strongly agree with Jason’s comments.

    I think one of the big “problems” to be solved in TDD is tools and, er, um, not sure what to call it, “supporting context”? By that I mean, we need better support for TDD at a lower level. In the same way that OO is now a language feature. TDD is a great idea, but it is still too awkward.

  • @Christopher You said, “the same way that OO is now a language feature.” What exactly do you think OO was “before” if not a language feature?

  • @Mike
    OO is an approach to programming. Just because a language has OO features, doesn’t mean your code is automatically OO. Likewise, you can use OO in languages that don’t explicitly, or entirely, support it (plain old C, or javascript even.) Microsoft’s AJAX tries pretty hard to make javascript feel more OO.
    It boils down to the use of OO techniques such as encapsulation, polymorphism, and inheritance.
    Similar arguments can be made about functional programming.

  • @Christopher Thank you for the clarification. I think we disagree about OO, but if we frame the discussion by your description I think your statement holds. I would add that you can use encapsulation, polymorphism, and inheritance and still not be OO.

    I continue to be puzzled by my observations of the .NET community’s desire to subject themselves to their tools. And now I will add “language features” to that list. While you can solve the problem of managing TDD complexity by adding additional “features” to the language, I submit that you can also solve the problem by using a stronger language.

  • “It was truly great to see people whom I thought had TDD down cold, experience some of the same problems…”
    “[TDD is] a practice which has been proven, through experience of many…”

    Was reading these two quotes from your post and suddenly thought: what if everyone is having problems with TDD? We each think we have a unique problem cause we are struggling and assume that everyone else is doing it right and has proven it successful? Kinda funny to think that JP and one or two Codebetter guys are the only ones who can get it to work :-)

    I’m getting more successful with TDD for my personal projects, but am having trouble translating it to the style of apps we normally do at work. I am confident it *can* work, but don’t know enough yet to help the team through an end-to-end TDD’d project.

    I think the reason some people struggle is that it is very difficult to gain proficiency with TDD without mentoring or a team working together on learning the technique. I’ve read all the books, done the examples, and packed my RSS reader full of TDDer’s blogs, but none of this helps when you get stuck on your specific problem and have no one to bounce ideas off.

    Please please please — when you start on with Jeremy and become the 4th person in the world to learn TDD, remember us un-mentored TDDers battling away and pass on everything you can :-) ;-)

  • So unfortunately the style you saw me doing was a style I’m experimenting with, not the style I’ve worked with for years. The BDD style is still pretty new and there isn’t a lot of direction out there (especially when the best source is deleted and gone). I have a feeling we did a disservice introducing that style in that context…

  • @Jason:
    Good stuff, thanks. That follows along mostly with what I’ve been doing, so it’s reassuring to know I’m not doing it totally wrong :)

    Great post! Subscribed!

    I plan on it. When Jeremy isn’t looking, I’ll be jotting notes down furiously. I may even secretly record the audio and screen so I can play it all back later in my mind. Shh, don’t tell him.

    @Jimmy (Bogard):
    Honestly, I don’t know how I ever did testing before that naming style. Sure, the context thing is still being sorted out, but doing it any other way usually ended in quick failure for me. TestSomething1, TestSomething2, TestSomething3 quickly grows tedious and seems futile.

    Remember, the point of this was to explore TDD, and trying to figure out BDD contexts is a part of that.

    When I saw my first nunit-spec report from my tests, the light turned on in my head. I never went back.

  • Chad, I gotta say that the one thing I appreciate about both you and Ray is the honesty. When you talk about your trials and tribulations with TDD, honestly I feel relief. I guess I am not the only one who has struggles with this stuff. I was THIS close to driving up from Houston to participate in the CodingDojo. I am really sorry I didn’t make it. Maybe I heard wrong, but there is going to be another one, right? I hope so.

  • Johan Söderberg

    Assert.AreEqual(y,x) is easier to read and follow, when you read from left to right you see what’s done before you get to the variables.

    Assert.That(x, Is.EqualTo(y)) is nastier, you need to remember what it’s doing when you get to the variables and you need to keep track of another thing (it actually does two things).

    Not that it matters much though. ;)


  • @Johan

    Underneath the covers, Assert.AreEqual calls Assert.That.

    I disagree that Assert.AreEqual reads better.

    Assert that x is equal to y.

    Assert are equal y x.

    You’re right though, it really doesn’t matter.

  • And since we’re in a C# 3.0 world now, don’t forget…