Preliminary thoughts on TDD with MSTest in Orcas
Naysawn, the PM for MSTest, asked that I take a look at the new unit testing features of Orcas, after he read my thoughts on the (sad) current state of MSTest. Unfortunately, I haven’t yet had a chance to actually fire up the Orcas VPC to play with it, but Naysawn wrote up a couple of posts describing some of the new features coming to MSTest in Orcas. So I figured I’d post some initial feedback on his posts as it relates to TDD…
From Naysawn’s First Post (with my comments below each item):
1 – Better Execution Times
Definitely a long time coming. Speed is a big thing to us serious TDD practitioners. Rapid feedback loops are key to staying productive using TDD.
2 – Run Tests
Hmm, sounds an awful lot like another tool of which I’m a huge fan; Jamie’s TestDriven.NET. Of course Microsoft doesn’t seem to be too much of a fan, but I’m not even getting into that mess in this post. Bottom line is that I’ve been using TestDriven.NET with Ctrl-T bound to the exact same commands Naysawn is talking about. So of course having this feature in MSTest is a good thing.
3 – Short Cut Keys to Run Tests
Being a serious keyboard junkie, this of course would be pretty important to me. Nuff said…
4 – Disable Deployment
This one is a biggie for me and probably would be one of my biggest complaints about the current implementation of MSTest. It not only tremendously slows down the testing process in general, but for unsuspecting developers (hmm, as opposed to suspicious developers? 😛 ), it can take up a huge amount of space over time. During some past engagements when trying to mentor TDD to folks using VSTS/MSTest, a couple months went by and they were wondering why they were running low on hard drive space. For a decent size application, doing a test run every 5-10 minutes, 5 days a week for a couple months can eat up a ton of space. Shadow copying is a decent idea and I understand what they were trying to accomplish, but at the very least, an option should exist to disable it.
5 – Inheritance of Tests
This is another important one for me. I’ve run into quite a few situations where test inheritance really leads to more maintainable tests and can cut down on the set up and customization of a suite of similar test fixtures. For instance, DB integration tests, where you need to refresh the database between tests, but you also need to override certain steps for customization of individual test fixtures. Of course favoring composition over inheritance is still preferrable, even in unit test code, but sometimes a simple base test class is simply all you need.
6 – Directly go to the Point of Failure
I actually really like the sound of this one (even though ReSharper has had something very similar for a while now). Just so long as I don’t have to move my hands all the way over to my mouse to jump to the stack trace. It sounds like a keyboard shortcut could be wired up for that though, so sounds good to me.
From Naysawn’s Second Post (with my comments below each item):
Gaining Access to Private Code under test
I can’t say I’m all that excited about this one because a) it starts sliding down that slippery slope of generating unit tests (more on that next) and b) frankly, I tend to not keep too many private methods around. Some of you may gasp at that, but I like to look for opportunities to extract logic out of private methods into separate public/internal classes hidden behind an interface that can be injected into the class under test. I won’t get too much into that right now, but let’s just say it can really make code more extensible. I like how Scott Bellware phrased it in his Seams, Not Components post. In the end, I guess having the ability to test private methods may have a small place, but ideally your classes and methods would be small enough to be driven and tested through its public interface using private methods as minor support if needed (i.e. readability, etc.).
Generating Unit Tests
Repeat after me. If I’m generating my unit tests, I’m not practicing Test-First Development. Hope that’s not too harsh, but no matter how you slice it, you can’t claim to be driving your code design through tests, if you’re generating your unit tests from existing code. Sure, I’ve heard folks say they like to stub out their interfaces/classes first and then generate their unit tests stubs from that. IMO, you’re tackling too much of the problem at once with that approach. If you’re really trying to leverage TDD to help you design your code, then a lot of the time, you don’t even know what your interfaces/classes are going to look like until you see them from the client’s perspective. And that is best done from within a unit test where you can express exactly how you want clients of your code to use the public API. It can’t be stated enough that TDD is supposed to be primarily a design tool.
Support for Device Projects
Having never done any work on embedded devices, can’t say much about this one. Although it is nice to know that it will be supported.
Automatic Cleanup of Test Results and Their Associated Deployments
I pretty much covered this in the #4 item above regarding the ability to disable deployments. Again, nice to see some flexibility come to this area of the product.
I definitely think this is a pretty large leap forward for MSTest and I’m happy to see Naysawn and the developer teams at Microsoft reaching out to the community more and more, and even more important, actually incorporating our feedback into the products. Big kudos for that…
With that said, as Jay Flowers points out, you’d be hard pressed to find anything better than MbUnit + TestDriven.NET + ReSharper right now. And frankly, I don’t really see that situation changing much even after Orcas is released. However, for those great many of organizations that have chosen or will end up choosing VSTS/MSTest (mostly for the wrong reasons, in my experience), these new features will bring them a little bit closer to the current ideal tools for TDD.