[TIP] Functional Testing of Desktop Applications

Michael Foord fuzzyman at voidspace.org.uk
Mon Mar 5 06:20:37 PST 2007

Grig Gheorghiu wrote:
> --- Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> This leads us into a bit of a debate about how much to mock out in
>> our 
>> unit tests. The most extreme testing doctrine says that you should
>> mock 
>> out *all* your dependencies when testing a unit - even dependencies 
>> within the same object (if you are testing method 'a' and it contains
>> a 
>> call to function or method 'b' you should mock 'b').
> I think this is a very extreme view of mocking. It would drive you
> crazy very fast if you started mocking dependencies within the same
> class.

Well - we do quite a lot of it. :-)

The principle being, that if you don't mock dependencies then you aren't 
really testing your units - but are testing other parts of the code *as 
well*, which ought to be adequately tested elsewhere.

What we want to test with the unittests are that the specific unit 
behaves as specified (with the tests providing the specification).

Perhaps this is quite 'extreme', but I'm still surprised that I haven't 
found other Python developers working in the same way. It's still 
reasonably orthodox 'eXtreme Programming', or at least that was how I 
was introduced to it.

This is what I meant in my blog post about the Python blogosphere not 
really exploring the areas of agile development that at least my 
colleague Andrzej was interested in (there are Ruby developers doing 
this for example).

> My view is that you should mock only when your code depends on external
> interfaces that generate random results (Web servers, XML-RPC servers,
> etc.). Generally speaking, at the I/O boundaries of your application,
> or when you deal with libraries/3rd party code over which you have no
> control. 
> It's also useful to mock the database layer, for several reasons:
> 1) speed 
> 2) consistency in getting back the results you need
> 3) simulating exceptions
> You can also achieve 1) and 2) with small test databases (in-memory or
> not). But it's very hard to achieve 3) without mocking.
>> When you have a short function that has several dependencies this can
>> make writing the tests a real chore. It *also* means that you don't 
>> catch errors at a higher level - where you mock you test adequately
>> what 
>> you *think* the wiring between your units is doing, but your mental 
>> model may be just as buggy as your code.
> You're right, if you overuse mocking, the test code can quickly get out
> of sync with the 'production' code.

Well, refactoring code means refactoring tests. This can sometimes be a 
drag, but the benefits of having extensive coverage make it worthwhile.

A too aggressive mocking style does tightly couple your tests to other 
parts of your code - but perhaps that is a code smell indicating that 
your units aren't as modular as they could be.

I think our mocking strategy veers erratically from too much to too 
little. It is still something we're exploring.

>> I'd be reluctant to add an *extra* level of testing (we already have
>> a 
>> three-to-one ratio between test and production code). We're moving 
>> towards a less extreme mocking policy I think though, making sure we 
>> always have some higher level tests that exercise units together and
>> not 
>> just in isolation.
> I think this is a good strategy.

This is still an open issue for us. Again - aggressive mocking leaves us 
with less tests at the intermediate level which test your wiring.


> Grig

More information about the testing-in-python mailing list