Comments on: Hosting an entire ASP.NET MVC request for testing purposes

By: Sean Scally

Sean Scally — Wed, 25 Jun 2008 16:23:05 +0000

@chadmyers

I’d really like to see the contents of each of those four test types that you mention; both what determines what goes in each type, and some examples of each.

There’s just not enough real-life examples on the web of such a breakdown in .NET projects.

By: erichexter

erichexter — Wed, 25 Jun 2008 04:26:06 +0000

While I appreciate the ability to test on the fly without IIS I think you could get similar results with the vs2008 dev webserver. I am not sure I understand why you would spend the time to do string assertions for the black box end-to-end tests, when you already have your view tests, which could check the state of objects in the webbrowser. Are you trying to eliminate the need to test multipage transactions in your view tests? At the end of the day shouldn’t the end-to-end tests drive the browser not the webserver?

Regardless of the motivation for these tests, I think it is pretty cool that you can do it.

By: Chad Myers

Chad Myers — Wed, 25 Jun 2008 03:35:23 +0000

@ScottGu

These are end-to-end ‘black box’ tests, not unit tests. I agree with you vis-a-vis unit tests and we do indeed do that (it’s still somewhat of a challenge due to the abundant use of the *Context classes — RoutContext, ViewContext, etc).

The reason I’m doing this here (instead of hitting a server) is so that they can be run in an automated fashion without the need for special setup on the box (i.e. on an Continuous Integration server that may not even have IIS installed).

Right now I can check out the code from my SVN repo, build it, and run the tests on an XP box without IIS installed.

We have layered tests that overlap a little, but we try to test different things at the various levels so as to avoid the “break a 1,000 tests when you refactor” problem.

** Unit tests – <2s secs, requires no special setup
** Integration Tests - <1 minute, requires database, maybe test data, some config files
** View Tests - >1 minute usually, involves just the views with minimal HttpContext setup, no HttpRuntime/ApplicationHost involves, WatiN and IE (soon FireFox)
** Black box end-to-end – >1 minute, involves the code above, includes the whole ASP.NET pipeline including URL routing, etc.

The black box tests are usually very specific about what they check and don’t involve a lot of HTML munging or big string comparison — check that XYZ exists, check that ABC is disabled, check that I get redirected to the login screen, etc.

By: ScottGu

ScottGu — Wed, 25 Jun 2008 03:17:55 +0000

I would typically recommend that your unit tests test the individual sub-components of your web application (routing, controllers, any custom filters you create, etc), and then only use an approach like above for end to end integration testing of everything (which should hopefully be far fewer tests).

I’m not sure I understand why you need to implement your ExecuteMvcUrl method though – can’t you just send standard URLs to the host and verify that the correct HTML is coming back? I’m not sure you want/need those end-to-end tests to be aware of which controller is handling what (your unit tests for your routing logic should cover those).

Potentially I’m missing the uber scenario though. Feel free to send me email (scottgu@microsoft.com) if you want to drill into it more and I’d be happy to help.

Thanks,

Scott