MVC Web Testing Strategies – verifying content

Some of the questions during the C4MVC presentation concerned how I liked to locate data displayed in the rendered HTML for validation purposes.  You have quite a few options for doing so, such as:

  • Traverse HTML directly through the DOM
  • Using existing semantic HTML information
  • Adding semantic HTML information
  • Wrapping data with specific HTML

I’ll dismiss the first option immediately, it’s quite brittle to traverse only the DOM to locate data on a screen.  If I have to have intimate knowledge of the entire HTML document rendered, the test will become far too dependent on the layout on the screen.  If I decide to move information out of a table and into a definition list, my test will likely break.

From there, I’m left with basically two options – semantic HTML, whether it’s existing or added, or wrapping each data piece with custom HTML.  In the presentation, two questions came up:

  • When locating a piece of information in a row in a table, why not use something more meaningful, such as the underlying entity’s ID to tag the data?
  • Again in the table, why not use semantic information on the row, instead of adding meaningless HTML?

Before we look at the different ways to verify content, let’s first look at what we want to verify.

The original content

In the scenario I highlighted in the screencast, I walk though editing some information, then verifying that data was changed correctly.  I still have tests for controllers, repositories and so on, but I want a high level scenario-based test to make sure the entire pipeline is working as expected.  We try and pull the information out of the HTML content rendered, using some sort of UI testing tool.  Without modifying the content, here’s what we have to work with:

<div id="main">
    <h2>Products</h2>
    <table>
        <thead>
            <tr>
                <td>Details</td>
                <td>Name</td>
                <td>Manufacturer</td>
                <td>Price</td>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td><a href="/Product/Edit/1">Edit</a></td>
                <td>Insignia® - 26" Class / 720p / 60Hz / LCD HDTV DVD Combo</td>
                <td>Insignia</td>
                <td>359.99</td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/2">Edit</a></td>
                <td>Insignia® - 19" Class / 720p / 60Hz / LCD HDTV</td>
                <td>Insignia</td>
                <td>189.99</td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/3">Edit</a></td>
                <td>Dynex® - 15" Class / 720p / 60Hz / LCD HDTV</td>
                <td>Dynex</td>
                <td>159.99</td>
            </tr>
        </tbody>
    </table>

There’s more rendered than just this HTML, but this is the piece that really matters.  We have some semantic CSS information, the DIV with an ID of “main”, but that’s about it.  In our CSS, we style our tables based on an ID-Element selector, but we could have just have easily used a class name on the table.  I’d like to verify that the price on the first product was changed correctly, but how do I verify this?  How do I make sure that the value in that one cell is the new price?

Choosing a strategy

I don’t want false positives or false negatives.  I don’t want to look for just any table or any table cell in the markup, I want really the piece just around “359.99”.  A few options to do so include:

  • Locating the outer DIV, finding the table, the first row, last cell, etc.
  • Apply a special class to the TD just where that price is shown
  • Add markup only around data-bound elements

When I first started doing UI testing, I assumed that I wanted to be as semantic and pure as possible, and not try to change my markup just for tests.  That idea lasted about an hour, and reality set in that in general, things not designed with testing in mind will be hard to test.  Hard to write tests, hard to maintain tests.

My next idea was to add information as needed, perhaps a class name to an existing surrounding element that might double for styling.  But I’ve moved completely away from that strategy as well, as styling and UI testing are two different concerns that should not mix.

Why keep these concerns separate?  Frankly, because both concerns have two completely different reasons to change.  Styling and behavior (such as IDs added for jQuery) are about design and interaction.  UI testing is about verifying site behavior.  Mixing the two concerns means my tests are more likely to break because I’ve changed the design.  But design can change without behavior changing, so I don’t want to couple my tests to the visual design.  Coupling UI tests to the site design is like unit testing using only the reflection API.

Instead, I’d rather couple my tests to something that better represents the view – and that would be the ViewModel.

Strongly-typed views, strongly-typed tests

We use strongly-typed views because using a dictionary just leads to brittleness and fear of change.  Strongly-typed views, along with the concept of the Thunderdome Principle, let us move past the issues of broken views and broken model binding.  Instead of writing black-box tests against our UI, we can take advantage of the strongly-typed views, and modify our HTML rendered to get access to the rendered ViewModel.  If you look at views from the point of view of “how to render a ViewModel”, then your UI tests can now talk in the language of the ViewModel.

But to help our UI tests out, we need to be able to locate information rendered from a ViewModel.  Looking at our original HTML, the most fool-proof, straightforward way to do this would be to add HTML as close to the rendered ViewModel as possible.  Let’s look at our view to see where this might be:

    <h2>Products</h2>
    <% var products = Model; %>
    
    <table>
        <thead>
            <tr>
                <td>Details</td>
                <td>Name</td>
                <td>Manufacturer</td>
                <td>Price</td>
            </tr>
        </thead>
        <tbody>
        <% foreach (var product in products) { %>
            <tr>
                <td><%= Html.ActionLink("Edit", "Edit", new { id = product.Id }) %></td>
                <td><%= product.Name %></td>
                <td><%= product.ManufacturerName %></td>
                <td><%= product.Price %></td>
            </tr>
        <% } %>
        </tbody>
    </table>

So what’s the ideal spot here?  On the table?  The table body?  Row?  Cell?  These are all possible…and all coupled to the design of the UI, rather than the shape of the ViewModel.  Instead of coupling to the design of the HTML, we can instead change every spot where we render the ViewModel directly to have extra HTML used only for UI tests.  In the presentation, I move towards this model:

<tbody>
<% var i = 0; %>
<% foreach (var product in products) { %>
    <tr>
        <td><%= Html.ActionLink("Edit", "Edit", new { id = product.Id }) %></td>
        <td><%= Html.Span(m => m[i].Name) %></td>
        <td><%= Html.Span(m => m[i].ManufacturerName) %></td>
        <td><%= Html.Span(m => m[i].Price) %></td>
    </tr>
<%
    i++;
} %>
</tbody>

In practice, I’ll use a similar concept to the Opinionated Input Builders Eric highlighted a while back.  Instead of input elements, I’ll solely render at the core, span tags.  Here is the rendered HTML:

<div id="main">
    <h2>Products</h2>
    <table>
        <thead>
            <tr>
                <td>Details</td>
                <td>Name</td>
                <td>Manufacturer</td>
                <td>Price</td>

            </tr>
        </thead>
        <tbody>
            <tr>
                <td><a href="/Product/Edit/1">Edit</a></td>
                <td><span id='_0__Name'>Insignia® - 26" Class / 720p / 60Hz / LCD HDTV DVD Combo</span></td>
                <td><span id='_0__ManufacturerName'>Insignia</span></td>
                <td><span id='_0__Price'>359.99</span></td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/2">Edit</a></td>
                <td><span id='_1__Name'>Insignia® - 19" Class / 720p / 60Hz / LCD HDTV</span></td>
                <td><span id='_1__ManufacturerName'>Insignia</span></td>
                <td><span id='_1__Price'>189.99</span></td>
            </tr>
            <tr>
                <td><a href="/Product/Edit/3">Edit</a></td>
                <td><span id='_2__Name'>Dynex® - 15" Class / 720p / 60Hz / LCD HDTV</span></td>
                <td><span id='_2__ManufacturerName'>Dynex</span></td>
                <td><span id='_2__Price'>159.99</span></td>
            </tr>
        </tbody>
    </table>

Notice that each data-bound output is wrapped with a span tag – and a special span tag at that.  The span tag has a unique ID that represents the expression used to render that piece of data.  The first product name shown comes from the expression “m => m[0].Name”.  Because I know the ViewModel used to render this view, I can use an expression combined with a UI testing tool to locate the rendering of any piece of information on the screen.

The downside of this strategy is the amount of HTML added to the final document render.  But since we’ve separated HTML used for styling and behavior, and HTML rendered strictly for testing, we can put a switch in our application so that for environments not doing UI testing, the span tag isn’t ever rendered!  We can then get the best of both worlds – non-obtrusive JavaScript on top of semantic HTML, and flip a switch to be able to get to any piece of information on the rendered HTML.

Because our UI tests also use the ViewModel types and expressions to locate the same rendered ViewModels that the views used to render, we no longer run into the issue where our UI tests become disconnected from the views rendered.  If we delete a property on our ViewModel, guess what, the UI test no longer compiles!

Wrapping it up

Developing a UI testing strategy isn’t the easiest thing in the world.  In our current project, I think we had three or four evolutions before we finally settled on a UI testing strategy we like.  And it’s still not perfect – we’re still looking at ways to make UI testing easy, expressive, and more valuable.  But gone are the days where a styling change broke our tests.  When our UI tests fail now, it’s more along the lines of a new required field, changing business rules and so on.

I don’t like mixing styling and behavior with locating data-bound UI elements, simply because both have very different reasons for change.  But by taking advantage of our strongly-typed views in our UI tests, we can render the ViewModel in such a way that makes it dirt-simple to locate individual pieces of our ViewModel on the final rendered page.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in ASP.NET MVC. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Ryan Roberts

    In before someone accuses you of reinventing webforms :)

    I have also found that strongly typed urls can also reduce the fragility of tests, as well as using the same test data model that the site is using for assertions.

  • Michael Wallett

    Hi Jimmy,

    Thanks for the great video and blog post. I’m interested in what tests you write for a single feature? My current thinking is that integration tests should be written at the highest level (UI). Write integration tests for controller actions if UI testing isn’t appropriate. Unit tests are not needed for controller actions if integration tests are in place. The domain model should be fully unit tested. What are your thoughts?

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Michael

    Right now, I’m really only writing tests for controllers if the controllers have interesting behavior. If all I’m doing is “return View(Model)”, meh. But if there is non-trivial behavior, we do put in unit tests simply because it’s easier to isolate failures of pure controller action logic that way.

  • http://MurrayOn.NET/ Mike Murray

    It might have been me that prompted this topic from my question in the screencast. I can agree with the span idea to uniquely identify each table cell so that you are not tied to a kludgey way of verifying the data change and subsequent presentation back to the user.

    However, the question I was trying to get to in the screencast is in regard to the use of row indices in those unique identifiers, instead of domain model id’s as unique identifiers. It seems like row indices would be tied too closely to the view concerns. What if I decided to sort differently in that view? But my test is testing if I can edit and reshow the changed values to the user, but it could now break because of an unrelated sorting concern in the view?

    Hopefully that question is more clear. Thanks again for the great screencast!

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Mike

    It’s usually not _too_ much of an issue, as we set up the data for each test. It’s a bit of a coupling, I agree, but we don’t always know the ID of each row, nor do we even know if each row corresponds to an entity. Sometimes it’s an intermediate object. The tradeoff for coupling to sorting rules in our situation worked out because of the disparity in what the rows represented.