I’m loving your exploration of testing and I am on the journey with you. But how about chucking the odd example in here and there?

Judging by your comments, people seem to be drawing their own conclusions. I myself wonder how your theory translates to practical application.

And finally thanks for writing these intriguing posts.

In other words, it will be acceptable, for example, to read a file to prepare data for a test case.
What do you think?

For example, suppose you have Widgets that are read/written through an IWidgetRepository, and you also have a WidgetProcessor that does some sort of magical processing on the widgets. Suppose your SUT exists at the same abstraction level as the WidgetProcessor and that it uses the WidgetProcessor. How would the SUT know what IWidgetRepository implementation to use to instantiate the WidgetProcessor?

If the SUT accepts an IWidgetRepository as a dependency, the seam has bubbled up to another level of abstraction. Now all objects that use the SUT must somehow resolve an IWidgetRepository implementation. The SUT’s dependence on IWidgetRepository is only indirect, it only serves to allow the instantiation of the WidgetProcessor.

On the other hand, if the SUT accepts an IWidgetProcessor as a dependency, an artificial seam has been created in the application. This seam adds a small degree of abstraction by hiding the WidgetProcessor’s dependencies, and some independence by letting you mock/stub the widget processor, but it isn’t much. The IWidgetProcessor will probably only have one implementation, and exists only for the purposes of registering WidgetProcessor in the DI container. I think this is a contributing factor to “mocks everywhere” and “interfaces for everything”, and can result in the types of tests which you mentioned previously.

There are also other solutions, like Service Locator and Factory patterns, but these are very similar to accepting the dependency as an argument. How is the factory or locator resolved? Do I have to write one for every environment (UT vs production)?

I’m not really sure how to modify the definition so as to prevent this, or if there is a definition that would do so. I do think that this is one of the biggest problems and areas of confusion when it comes to defining a UT strategy. Maybe someday I’ll have some sort of epiphany on how to solve it.

1. They run fast.
2. They help localise problems.

He goes on to say that a unit test that takes 1/10 sec to run is a SLOW unit test. The slower the tests, the less likely is it that a developer is going to keep running them. Which defeats the purpose.

Reducing the “integration” scope of a unit test from several classes to a few or a single class will speed up these tests. You can also see that DB or network access will totally kill the speed (and reduce localisation of problems).

I think his point about speed is very crucial.

Write the test that needs to be written.

That might be my favorite definition – things like a repository crosses a stream, which I’d want to stub most of the time in a unit test. But peer objects, objects of the same abstraction, I’d want the real-deals in play.

This doesn’t require duplication of tests or mocking data access in unit tests.

Say we had two methods to test. “CalculateCartCheckoutPrice” is a complex calculation using a number of domain objects, and doesn’t do any data access. “SaveCartContentsToDatabase” mainly does data access (saving the domain objects to a database) and not much logic.

“CalculateCartCheckoutPrice” tests would be considered unit tests, but “SaveCartContentsToDatabase” tests would be integration tests. The integration tests are then mainly testing the design of the database (tables, stored procedures) and data interface. The unit tests are separately testing the logic, and are where you want faster iterations of tests.

If the web service is down or broken, that at least alerts someone to the issue. 9 out of 10 times I bet the exception is something generic and has no meaning if someone looked at it. Which means you write the exception ABCCompanyWebServiceDownAfter5Retries, vs the Remote Host Down after you get tired of looking up the error or decide to throw an “inconclusive”. I even bet that there is 0 retry logic in 99% of the code out there. Also if you have a single if statement in a stored proc, its code. So a simple “update if exists, insert if it doesn’t” stored proc now contains logic. So it has to be tested to make sure everything is working as expected.

I think the database, file system, and network could be better classified as external systems. Web services and cooperative processes seem to be notable omissions that fall into this category. External systems are one of the areas where we usually want to introduce seams into the application so that we do not depend on the external system in order to verify ours. We introduce seams into applications so that we can achieve new levels of abstraction and independence.

My best shot at defining a unit test would be: A unit test is a test that verifies one unit of functionality at a single level of abstraction, but may specify behavior at a seam. Once a test crosses a seam, it becomes an integration test.

A unit of functionality could be a method, class, user story, etc…. The SUT may use objects at multiple levels of abstraction to do its job, but the test is only aware of one of them. Mocks/stubs are allowed as stand-ins only at the seam boundary. The test’s abstraction level will probably be the same abstraction level at which the SUT exists. I’m not sure if that is necessary though.

I think this also speaks to the problem you mentioned in your previous post. The tests were aware of both the desired results (high abstraction level) and the implementation details (low abstraction level) of the SUT. If the test only used mocks/stubs at seams, refactoring would probably be easier, as long as the APIs to the seams are less likely to change.

I understand that, but it is hard to swing a dead cat and not come across similar posts with sentiments like “I’ve been doing TDD for three years, and most of the time, I’ve been doing it wrong.”

I think there’s a reason for that.