Effective Tests: Custom Assertions

In our last installment, we took a look at using Auto-mocking Containers as a way of reducing coupling and obscurity within our tests. This time, we’ll take a look at another technique which aids in preventing obscurity caused by complex assertions: Custom Assertions.

Custom Assertions

As discussed earlier in the series, executable specifications are a model for our requirements. Whether serving as a guide for our implementation or as documentation for existing behavior, specifications should be easy to understand. Assertion implementation is one of the areas that can often begin to obscure the intent of our specifications. When standard assertions fall short of expressing what is being validated clearly and concisely, they can be replaced with Custom Assertions. Custom assertions are domain-specific assertions which encapsulate complex or obscure testing logic within intention-revealing methods.

Consider the following example which validates that the items returned from a ReviewService class are sorted in descending order:

    public class when_a_customer_retrieves_comment_history : WithSubject<ReviewService>
    {
        const string ItemId = "123";
        static IEnumerable<Comment> _comments;

        Establish context = () => For<IItemRepository>()
                                      .Setup(x => x.Get(ItemId))
                                      .Returns(new Item(new[]
                                                            {
                                                                new Comment("comment 1", DateTime.MinValue.AddDays(1)),
                                                                new Comment("comment 2", DateTime.MinValue.AddDays(2)),
                                                                new Comment("comment 3", DateTime.MinValue.AddDays(3))
                                                            }));

        Because of = () => _comments = Subject.GetCommentsForItem(ItemId);
      
        It should_return_comments_sorted_by_date_in_descending_order = () =>
            {
                Comment[] commentsArray = _comments.ToArray();
                for (int i = commentsArray.Length - 1; i > 0; i--)
                {
                    if (commentsArray[i].TimeStamp > commentsArray[i - 1].TimeStamp)
                    {
                        throw new Exception(
                            string.Format(
                                "Expected comments sorted in descending order, but found comment \'{0}\' on {1} after \'{2}\' on {3}",
                                commentsArray[i].Text, commentsArray[i].TimeStamp, commentsArray[i - 1].Text,
                                commentsArray[i - 1].TimeStamp));
                    }
                }
            };
    }

 

While the identifiers used to describe the specification are clear, the observation’s implementation contains a significant amount of verification logic which makes the specification more difficult to read. By moving the verification logic into a custom assertion which describes what is expected, we can clarify the specification’s intent.

When developing on the .Net platform, Extension Methods provide a nice way of encapsulating assertion logic while achieving an expressive API. The following listing shows the same assertion logic contained within an extension method:

    public static class CustomAssertions
    {
        public static void ShouldBeSortedByDateInDescendingOrder(this IEnumerable<Comment> comments)
        {
            Comment[] commentsArray = comments.ToArray();
            for (int i = commentsArray.Length - 1; i > 0; i--)
            {
                if (commentsArray[i].TimeStamp > commentsArray[i - 1].TimeStamp)
                {
                    throw new Exception(
                        string.Format(
                            "Expected comments sorted in descending order, but found comment \'{0}\' on {1} after \'{2}\' on {3}",
                            commentsArray[i].Text, commentsArray[i].TimeStamp, commentsArray[i - 1].Text,
                            commentsArray[i - 1].TimeStamp));
                }
            }
        }
    }

 

Using this new custom assertion, the specification can be rewritten to be more intention-revealing:

    public class when_a_customer_retrieves_comment_history : WithSubject<ReviewService>
    {
        const string ItemId = "123";
        static IEnumerable<Comment> _comments;

        Establish context = () => For<IItemRepository>()
                                      .Setup(x => x.Get(ItemId))
                                      .Returns(new Item(new[]
                                                            {
                                                                new Comment("comment 1", DateTime.MinValue.AddDays(1)),
                                                                new Comment("comment 2", DateTime.MinValue.AddDays(2)),
                                                                new Comment("comment 3", DateTime.MinValue.AddDays(3))
                                                            }));

        Because of = () => _comments = Subject.GetCommentsForItem(ItemId);

        It should_return_comments_sorted_by_date_in_descending_order = () => _comments.ShouldBeSortedByDateInDescendingOrder();
    }

Verifying Assertion Logic

Another advantage of factoring out complex assertion logic into custom assertion methods is the ability to verify that the logic actually works as expected. This can be particularly valuable if the assertion logic is reused by many specifications.

The following listing shows positive and negative tests for our custom assertion:

    public class when_asserting_unsorted_comments_are_sorted_in_descending_order
    {
        static Exception _exception;
        static List<Comment> _unsortedComments;

        Establish context = () =>
            {
                _unsortedComments = new List<Comment>
                                        {
                                            new Comment("comment 1", DateTime.MinValue.AddDays(1)),
                                            new Comment("comment 4", DateTime.MinValue.AddDays(4)),
                                            new Comment("comment 3", DateTime.MinValue.AddDays(3)),
                                            new Comment("comment 2", DateTime.MinValue.AddDays(2))
                                        };
            };

        Because of = () => _exception = Catch.Exception(() => _unsortedComments.ShouldBeSortedByDateInDescendingOrder());

        It should_throw_an_exception = () => _exception.ShouldBeOfType(typeof(Exception));
    }

    public class when_asserting_sorted_comments_are_sorted_in_descending_order
    {
        static Exception _exception;
        static List<Comment> _unsortedComments;

        Establish context = () =>
        {
            _unsortedComments = new List<Comment>
                                        {
                                            new Comment("comment 4", DateTime.MinValue.AddDays(4)),
                                            new Comment("comment 3", DateTime.MinValue.AddDays(3)),
                                            new Comment("comment 2", DateTime.MinValue.AddDays(2)),
                                            new Comment("comment 1", DateTime.MinValue.AddDays(1))
                                        };
        };

        Because of = () => _exception = Catch.Exception(() => _unsortedComments.ShouldBeSortedByDateInDescendingOrder());

        It should_not_throw_an_exception = () => _exception.ShouldBeNull();
    }

Conclusion

This time, we examined a simple strategy for clarifying the intent of our specifications involving the movement of complex verification logic into custom assertions methods. Next time we’ll take a look at another strategy for clarifying the intent of our specifications which also serves to reduce test code duplication and test-specific code within production.

About Derek Greer

Derek Greer is a consultant, aspiring software craftsman and agile enthusiast currently specializing in C# development on the .Net platform.
This entry was posted in Uncategorized and tagged , . Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Dennis Doomen

    ddd

  • Dennis Doomen

    Great article series, but I wonder why you’re not suggesting using one of the many open-source assertion libraries such as my own Fluent Assertions (http://fluentassertions.codeplex.com/documentation)

    • http://derekgreer.lostechies.com Derek Greer

      I’ve tried to keep the series focused on the concepts involved in writing effective automated tests, not on the use of any particular testing library or framework.  That aside, your library seems to be primarily focused on expressing assertions in a fluent-style which isn’t really the topic of this article.

      I personally haven’t found the need for using a separate assertion library since my testing framework of choice is MSpec which already provides a set of Should extensions, but I would certainly encourage people using frameworks which don’t provide such extensions to consider using a separate assertion library.  I’m not familiar with Fluent Assertions, but I have used Eric Hexter’s Should Assertion Library which I recommend to those who like MSpec’s assertion style.

  • Pingback: The Morning Brew - Chris Alcock » The Morning Brew #871