MVC Web Testing Strategies – verifying content
Some of the questions during the C4MVC presentation concerned how I liked to locate data displayed in the rendered HTML for validation purposes. You have quite a few options for doing so, such as:
- Traverse HTML directly through the DOM
- Using existing semantic HTML information
- Adding semantic HTML information
- Wrapping data with specific HTML
I’ll dismiss the first option immediately, it’s quite brittle to traverse only the DOM to locate data on a screen. If I have to have intimate knowledge of the entire HTML document rendered, the test will become far too dependent on the layout on the screen. If I decide to move information out of a table and into a definition list, my test will likely break.
From there, I’m left with basically two options – semantic HTML, whether it’s existing or added, or wrapping each data piece with custom HTML. In the presentation, two questions came up:
- When locating a piece of information in a row in a table, why not use something more meaningful, such as the underlying entity’s ID to tag the data?
- Again in the table, why not use semantic information on the row, instead of adding meaningless HTML?
Before we look at the different ways to verify content, let’s first look at what we want to verify.
The original content
In the scenario I highlighted in the screencast, I walk though editing some information, then verifying that data was changed correctly. I still have tests for controllers, repositories and so on, but I want a high level scenario-based test to make sure the entire pipeline is working as expected. We try and pull the information out of the HTML content rendered, using some sort of UI testing tool. Without modifying the content, here’s what we have to work with:
<div id="main"> <h2>Products</h2> <table> <thead> <tr> <td>Details</td> <td>Name</td> <td>Manufacturer</td> <td>Price</td> </tr> </thead> <tbody> <tr> <td><a href="/Product/Edit/1">Edit</a></td> <td>Insignia® - 26" Class / 720p / 60Hz / LCD HDTV DVD Combo</td> <td>Insignia</td> <td>359.99</td> </tr> <tr> <td><a href="/Product/Edit/2">Edit</a></td> <td>Insignia® - 19" Class / 720p / 60Hz / LCD HDTV</td> <td>Insignia</td> <td>189.99</td> </tr> <tr> <td><a href="/Product/Edit/3">Edit</a></td> <td>Dynex® - 15" Class / 720p / 60Hz / LCD HDTV</td> <td>Dynex</td> <td>159.99</td> </tr> </tbody> </table>
There’s more rendered than just this HTML, but this is the piece that really matters. We have some semantic CSS information, the DIV with an ID of “main”, but that’s about it. In our CSS, we style our tables based on an ID-Element selector, but we could have just have easily used a class name on the table. I’d like to verify that the price on the first product was changed correctly, but how do I verify this? How do I make sure that the value in that one cell is the new price?
Choosing a strategy
I don’t want false positives or false negatives. I don’t want to look for just any table or any table cell in the markup, I want really the piece just around “359.99”. A few options to do so include:
- Locating the outer DIV, finding the table, the first row, last cell, etc.
- Apply a special class to the TD just where that price is shown
- Add markup only around data-bound elements
When I first started doing UI testing, I assumed that I wanted to be as semantic and pure as possible, and not try to change my markup just for tests. That idea lasted about an hour, and reality set in that in general, things not designed with testing in mind will be hard to test. Hard to write tests, hard to maintain tests.
My next idea was to add information as needed, perhaps a class name to an existing surrounding element that might double for styling. But I’ve moved completely away from that strategy as well, as styling and UI testing are two different concerns that should not mix.
Why keep these concerns separate? Frankly, because both concerns have two completely different reasons to change. Styling and behavior (such as IDs added for jQuery) are about design and interaction. UI testing is about verifying site behavior. Mixing the two concerns means my tests are more likely to break because I’ve changed the design. But design can change without behavior changing, so I don’t want to couple my tests to the visual design. Coupling UI tests to the site design is like unit testing using only the reflection API.
Instead, I’d rather couple my tests to something that better represents the view – and that would be the ViewModel.
Strongly-typed views, strongly-typed tests
We use strongly-typed views because using a dictionary just leads to brittleness and fear of change. Strongly-typed views, along with the concept of the Thunderdome Principle, let us move past the issues of broken views and broken model binding. Instead of writing black-box tests against our UI, we can take advantage of the strongly-typed views, and modify our HTML rendered to get access to the rendered ViewModel. If you look at views from the point of view of “how to render a ViewModel”, then your UI tests can now talk in the language of the ViewModel.
But to help our UI tests out, we need to be able to locate information rendered from a ViewModel. Looking at our original HTML, the most fool-proof, straightforward way to do this would be to add HTML as close to the rendered ViewModel as possible. Let’s look at our view to see where this might be:
<h2>Products</h2> <% var products = Model; %> <table> <thead> <tr> <td>Details</td> <td>Name</td> <td>Manufacturer</td> <td>Price</td> </tr> </thead> <tbody> <% foreach (var product in products) { %> <tr> <td><%= Html.ActionLink("Edit", "Edit", new { id = product.Id }) %></td> <td><%= product.Name %></td> <td><%= product.ManufacturerName %></td> <td><%= product.Price %></td> </tr> <% } %> </tbody> </table>
So what’s the ideal spot here? On the table? The table body? Row? Cell? These are all possible…and all coupled to the design of the UI, rather than the shape of the ViewModel. Instead of coupling to the design of the HTML, we can instead change every spot where we render the ViewModel directly to have extra HTML used only for UI tests. In the presentation, I move towards this model:
<tbody> <% var i = 0; %> <% foreach (var product in products) { %> <tr> <td><%= Html.ActionLink("Edit", "Edit", new { id = product.Id }) %></td> <td><%= Html.Span(m => m[i].Name) %></td> <td><%= Html.Span(m => m[i].ManufacturerName) %></td> <td><%= Html.Span(m => m[i].Price) %></td> </tr> <% i++; } %> </tbody>
In practice, I’ll use a similar concept to the Opinionated Input Builders Eric highlighted a while back. Instead of input elements, I’ll solely render at the core, span tags. Here is the rendered HTML:
<div id="main"> <h2>Products</h2> <table> <thead> <tr> <td>Details</td> <td>Name</td> <td>Manufacturer</td> <td>Price</td> </tr> </thead> <tbody> <tr> <td><a href="/Product/Edit/1">Edit</a></td> <td><span id='_0__Name'>Insignia® - 26" Class / 720p / 60Hz / LCD HDTV DVD Combo</span></td> <td><span id='_0__ManufacturerName'>Insignia</span></td> <td><span id='_0__Price'>359.99</span></td> </tr> <tr> <td><a href="/Product/Edit/2">Edit</a></td> <td><span id='_1__Name'>Insignia® - 19" Class / 720p / 60Hz / LCD HDTV</span></td> <td><span id='_1__ManufacturerName'>Insignia</span></td> <td><span id='_1__Price'>189.99</span></td> </tr> <tr> <td><a href="/Product/Edit/3">Edit</a></td> <td><span id='_2__Name'>Dynex® - 15" Class / 720p / 60Hz / LCD HDTV</span></td> <td><span id='_2__ManufacturerName'>Dynex</span></td> <td><span id='_2__Price'>159.99</span></td> </tr> </tbody> </table>
Notice that each data-bound output is wrapped with a span tag – and a special span tag at that. The span tag has a unique ID that represents the expression used to render that piece of data. The first product name shown comes from the expression “m => m[0].Name”. Because I know the ViewModel used to render this view, I can use an expression combined with a UI testing tool to locate the rendering of any piece of information on the screen.
The downside of this strategy is the amount of HTML added to the final document render. But since we’ve separated HTML used for styling and behavior, and HTML rendered strictly for testing, we can put a switch in our application so that for environments not doing UI testing, the span tag isn’t ever rendered! We can then get the best of both worlds – non-obtrusive JavaScript on top of semantic HTML, and flip a switch to be able to get to any piece of information on the rendered HTML.
Because our UI tests also use the ViewModel types and expressions to locate the same rendered ViewModels that the views used to render, we no longer run into the issue where our UI tests become disconnected from the views rendered. If we delete a property on our ViewModel, guess what, the UI test no longer compiles!
Wrapping it up
Developing a UI testing strategy isn’t the easiest thing in the world. In our current project, I think we had three or four evolutions before we finally settled on a UI testing strategy we like. And it’s still not perfect – we’re still looking at ways to make UI testing easy, expressive, and more valuable. But gone are the days where a styling change broke our tests. When our UI tests fail now, it’s more along the lines of a new required field, changing business rules and so on.
I don’t like mixing styling and behavior with locating data-bound UI elements, simply because both have very different reasons for change. But by taking advantage of our strongly-typed views in our UI tests, we can render the ViewModel in such a way that makes it dirt-simple to locate individual pieces of our ViewModel on the final rendered page.