Last night at
.Net Developer Network Steve Strong (
@srstrong) gave us a presentation on
Pex & Moles which has been created by
Microsoft Research.
I first heard of Pex a while back (possibly 2 years ago now) and whilst I liked the look of it dependencies were a problem and so I didn’t pursue it any further, so I was especially interested to see Steve’s presentation and see how it had changed.
Pex
Pex is a Visual Studio add in for VS2008 and VS2010 which will create
parameterised unit tests for your code, it aggressively (I’ll come back to that in a minute) goes through your code trying to create sufficient tests to attempt to get 100% code coverage.
Pex is fairly non-partisan about what unit test framework it uses and will just as happily use NUnit, MbUnit, etc as it will MSTest.
Pex will generate tests, including failing tests if the code doesn't work the way it thinks it should work. These are created as normal tests that you can run individually just as you normally would. If you have created any tests yourself then Pex will use those tests to improve the quality of the tests that it creates.
Although Pex will do a fairly good job of test creation you should review the values it uses for the tests to ensure that you are happy about the values in case it has missed something or is using data which is just plain wrong; Pex does offer a host of functionality so that you can influence the data it creates restricting ranges, min or max values, etc.
Pex works by instrumenting the IL generated for your code so that it can track the various paths that are executed allowing it to then feed in various values it generates to exercise your code, these include some default values such as null, MinValue, 0, etc.
As I mentioned earlier Pex goes through your code aggressively, this means that it looks at your tests, your code, any libraries you use (including standard .Net assemblies) and attempts to follow as many branches through the code as possible. On a practical note Pex may fail to be able to create tests to give 100% coverage if it cannot satisfy itself that it has exercised a branch in the code sufficiently, so the more complex the code the less likely Pex will manage to create the tests. Because of this Pex is not suitable for system testing as the possible branches across a system will be just too many for it to create any meaningful tests.
Moles
Ok so you’ve got this great tool that can generate you tests but you need to mock dependencies such as a database so what do you do? You could try using
Rhino Mocks or
Moq but you run straight into the problem that Pex will attempt to go through the entire mocking library assembly and test that.
[Edit:] Another reason that Rhino & Moq are likely to cause problems is that because they generate new classes under the covers Steve was unsure exactly what Pex would do about those particular classes [Edit]
So the Pex team came up with Moles, which is not really a mocking library but a stubbing library, to allow you to stub your dependencies. The way it does it is the usual sub typing method which does mean that you can’t stub the normal suspects (sealed classes, static classes, value type, etc) but Moles will go through an entire assembly and generate a mole for each type in that assembly that it can stub, included generic types as well, without you having to do it all yourself manually.
What you can then do is determine the behaviour of a mole by assigning delegates to the properties/methods of that stub and because the mole code is so simply pex is unlikely to ‘choke’ on it when observing your code.
Moles also come with behaviors (yes it is spelt correctly) and using these behaviors you can determine what the code should do for any properties/methods that you don’t explicitly provide an implementation for. It would appear that moles work in a similar way to
TypeMock and as such they are able to intercept calls in your code that normally would not be accessible to normal mocking libraries.
So are they any good?
What I’ve mentioned here only really scratches the surface of what Steve covered and you need to see Pex in action to appreciate what they can do and functionality that is available.
Steve did say that he believed you would end up using Pex to generate tests for your code
and hand crafting specific tests to cover business logic that Pex just can’t get e.g. behavioural tests.
I will say that based on Steve’s presentation I will be taking another look at Pex & Moles for my code.
Many thanks to Steve for the presentation and Guy as always for organising the evening.