UI Automation tools snake oil?

Michael Feathers posted a thoughtful piece describing the general problems of UI testing tools and the industry in general.  In general, I’d agree here.  Automation tool vendors, as with almost every tool vendor out there, are eager to solve perceived software development problems.  Unfortunately, these tools usually only address symptoms of larger problems.

With UI testing tools, this is no different.  When we started out authoring UI tests for a large MVC application, we had a few large goals in mind.  The tests needed to be:

  • Stable
  • Understandable
  • Maintainable
  • Valuable

To achieve these goals, we needed to do exactly what we do when we write code using TDD, design for testability.  I won’t go into everything we did, there’s already a whole screencast on that.  However, Michael does bring up some key points:

  • UI tests should test the UI layer
  • Architect your system so that you can test all of your business logic separate from the UI layer

That’s what we’ve done, and it’s been very successful.  We have UI tests, around 500 of them, and we’re not staying up until all hours of the night keeping them working.  We also have subcutaneous tests, that work in a layer right below the UI layer, that allow us to send a UI message into our system and check what comes out on the other side, whether it’s business rule violations or entities modified.

All of our testability enhancements came from design for testability.  We use strongly-typed views.  We don’t have any business/domain logic in controllers, only UI controller logic.  Separating your architecture into layers isn’t for portability, it’s for separation of concerns.  Once we started designing for testability in the large, and not just the unit level, we were able to maximize the value of automated tests, both at the UI, integration and unit level.

If anything, this is another example of the need for a developer to work on their spidey sense for vendor product demos.  We rail on MS for drag-and-drop demos, but it’s every tool vendor that employs this technique.  Just like every other product sold in the world, it’s up to the buyer to be critical of the true value a tool can bring.

Related Articles:

Post Footer automatically generated by Add Post Footer Plugin for wordpress.

About Jimmy Bogard

I'm a technical architect with Headspring in Austin, TX. I focus on DDD, distributed systems, and any other acronym-centric design/architecture/methodology. I created AutoMapper and am a co-author of the ASP.NET MVC in Action books.
This entry was posted in Testing. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://jason.pettys.name Jason Pettys

    Hey Jimmy — thanks for responding. I found Feathers’s article and thought he spoke the truth, but literally the first thought I had was, “I wonder what Jimmy would think of this.” Then I found your comment on his article :)

    Thanks for the info.

  • Arnis L.

    I’m glad to hear success story. UI tests are quite hard to handle.