Five rules for writing effective UI tests

For about 3 years, I wrote absolutely horrible UI tests.  These tests worked great for a while, but like most people that tried their hands at UI tests, found them to be utterly brittle and unmaintainable in the face of change.

And, like most people, I would mark these tests explicit or ignored, until I ditched most or all of them.  I’ve heard this story over and over again, from folks that wrote lots and lots of UI tests, only to see their investment go to waste as the time and effort to maintain these often-breaking tests far outweighs any benefit gained from full, end-to-end UI tests.  I gave some presentations on UI testing, but two things I’d like to make absolutely clear:

  • Maintainable UI tests are absolutely possible
  • Automated UI tests are absolutely possible

I know this because our team did this.  It was quite a bit of work to get to that point, but once we did, we had a great set of concepts on how we can continue to do so in the future.

In a recent show-and-tell session with another team that I really wish would blog about their efforts (they know who they are ;) , it became even more clear to me exactly why we had such success with our UI tests.  I didn’t have all the pieces in my head, and lots of our success was completely accidental, but we settled on a set of rules that no matter what our external user interface technology, these rules led to maintainable, effective UI tests.

Rule 1: Strongly-typed views

Whether it’s ASP.NET MVC, WebForms, Silverlight or whatever, the views need to be built from a model type.  It doesn’t really matter how that model type is built, CQRS, SQL projection, LINQ projection, AutoMapper or what, but we got the most mileage out of creating models specific for each view.  That type represented the information to show on that screen.

Besides being a great source for metadata about the view, a view-specific model, or “view model” becomes a specification for how the view should render.  Our views are very convention-based, where things like the presence of a “RequiredField” attribute not only instructs the validation framework to kick in.

There’s the magic string problem with web frameworks such as ASP.NET MVC, that strongly-typed views eliminate.  But step 1 – strongly-typed views.

Rule 2: Expression-based UI element generation

Another big must.  Every UI element needs to be generated from a C# expression used to output the display or edit field.  With ASP.NET MVC, it means doing something like “Html.DisplayFor” or “Html.EditorFor(m => m.FirstName)”.

We want this because strongly typed views plus expression based UI element generation gives us a very, very powerful tool to build very intelligent views.  These intelligent views are critical to easily designing for UI tests.  Using the expression gives us a metric ton of metadata information from which we can build convention-based UI element generators, including:

  • Class type/attributes
  • Property type
  • Property name
  • Attributes
  • Property value

All this information gives us all we need to easily build UIs that don’t break and are easy to test.

Rule 3: Metadata-driven UI element identifiers

When it comes to the mechanics of UI testing, you have to figure out how the UI framework will locate edit and display fields in your specific UI platform.  For the web, we chose to use the ID HTML element for both display and edit fields.  Display fields were wrapped in <span> tags with an ID value matching the property name of the view model property.

If we coded “Html.DisplayFor(m => m.FirstName)”, this would render “<span id=’FirstName’>Bob</span>”.  In the edit fields, the property name shows up in both the NAME and ID attributes of the input field.

Since the HTML generation includes model-based identifiers, this greatly helps locating these elements in UI automation frameworks.  Magic strings get eliminated from both your views and your UI tests.

In other frameworks, such as Silverlight and WinForms, element location is done by grabbing the actual field’s name in the form class.  In that case, something like “TextBox1” is useless.  We want the element identifiers to be driven off of the property name metadata of our view models.

Rule 4: Expression-based element locators in tests

When it comes to building our UI tests, we want to use the EXACT SAME expressions in our UI tests that we find in our views.  If I code “Html.EditorFor(m => m.FirstName)” in my view, I need to see something like “Form.TextBox(m => m.FirstName, “Bob”)” in my UI tests.  The exact same view model type and the exact same expression are used to generate the UI element, as is used to locate the UI element.

When my UI element generation and location logic are the same, and bound by view model metadata, I no longer have to worry about broken UI tests because of changing my view.  If I remove a member from the screen, I delete that property from my view model, and now I need to update my UI test, all before I can commit the change!

I really can’t stress enough how important this rule is.  Without it, you’re really spending too much time futzing around with broken UI tests, or just not getting as much mileage as you could have.

Rule 5: Set up context using existing business logic

If you need to set up data, and you WILL need to set up data, don’t go through any back doors to set things up.  Don’t build new data access layers, and don’t go through custom-built back doors to build state that doesn’t go through the normal domain logic.

Whatever your domain layer is, use it to set up your contexts.  Don’t design workarounds to get things in a state that isn’t normally possible.  If you look for usages of a type or member and it’s only used in tests, you’re asking for brittle tests.

Wrapping it up

Effective, maintainable UI tests are definitely possible.  They can be written in such a way that it doesn’t require lots of babysitting or dedicated personnel to maintain.  They can even be automated to run after every commit.  They can provide value, catch bugs, and expose all those unintended consequences from changes that complex applications tend to have.

It’s all possible, but only if the UI is designed with tests in mind.  I’m sure these aren’t the only ways to create maintainable UI tests, as I know the Dovetail folks also have a lot of innovation in that area, but these rules certainly worked for us.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in Testing. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Steve Hebert

    Have you done anything with rehydration testing? (render a partial or template to a stream and then model bind to a depersisted object to test wire up?).

    Would love to test this path in the lightest manner possible.

  • http://biasecurities.com Jim Geurts

    This sounds very interesting. I would like to hear more about how you test complex views, though.

    From what I took, rather than passing model objects directly to the view model, you setup fields on the view model that “mapped” to the model object fields. What about cases where there are collections of objects associated w/ the model object, creating complex object graphs? How are those tested?

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @Steve

    No, I don’t think so. Although if you used Spark, it might be a lot easier.

    @Jim

    It’s always a simple ViewModel object that gets built. Simple as in no logic, but represents everything on the screen. We’ve had screens with hundreds of fields, nested collections (not our choice), and this approach still worked as everything still revolved around expressions and view model types.

  • http://mrdustpan.blogspot.com/ MrDustpan

    This is very interesting, something I’ve really wanted to figure out how to do. Any chance of an example (even just a simple one) of what this looks like in action?

  • http://www.lostechies.com/members/bogardj/default.aspx bogardj

    @MrDustpan

    Follow that link in the post, it has a video with slides & code.

  • dario-g

    How about displaying fields according to permissions? For example one role can see field A, other field A,B and next only B, etc. Do you create views for every combination of permissions?

  • John B

    My problem with all the browser-automation tools is that they seem to work for any website as long as that website doesn’t use any javascript. As soon as you need to get the browser to wait for an ajax call or a jQuery .show(), you’re left to your own devices, and googling for answers only seems to give me claims of workarounds that don’t even kind of work.

    How do you deal with tricking these tools into waiting long enough for javascript/ajax without littering the tests with Thread.Sleep()?

  • aneal

    Hi jimmy,
    This article is really helpful for beginner SDET guys like me ! Thanks for the same !!!
    I already have the UI in place and it was not build with the intention that automated testing would be used.

    Please provide me tips for writing automated test cases in such a situation.

    thanks,
    aneal.

  • Pingback: Software Testing Articles #1

  • Tom Hunter

    Hi Jimmy, great article – thanks for the tips. I’m currently building an ASP.NET MVC 4 web application and I wondered if you had any recommendations on UI testing tools in 2013 (preferably free!).

    Thanks,
    Tom