[TIP] Functional Testing of Desktop Applications

[Michael Foord]

Laura Creighton wrote:
> In a message of Mon, 05 Mar 2007 00:01:35 GMT, Michael Foord writes:
> <snip>
>   
>> This leads us into a bit of a debate about how much to mock out in our 
>> unit tests. The most extreme testing doctrine says that you should mock 
>> out *all* your dependencies when testing a unit - even dependencies 
>> within the same object (if you are testing method 'a' and it contains a 
>> call to function or method 'b' you should mock 'b').
>>
>> When you have a short function that has several dependencies this can 
>> make writing the tests a real chore. It *also* means that you don't 
>> catch errors at a higher level - where you mock you test adequately what 
>> you *think* the wiring between your units is doing, but your mental 
>> model may be just as buggy as your code.
>>     
>
> A different, related problem.  The idea is that your tests stay live
> and you run them all periodically to make sure nothing breaks.  Do
> this properly, and write a thing like pypy (and py.test) and you
> will find that a run of all your tests takes 3-4 hours. :-(  Thus
> the most extreme doctrine above only works on "small-enough" projects.
>
> Deciding what to do when your project is no longer "small enough" is
> hard.
>
>   
"Thus the most extreme doctrine above only works on "small-enough" 
projects."

I don't understand what you mean: mocking makes your tests faster - not 
slower. Do you mean that having full test coverage is only possible / 
worthwhile on smaller projects ?

Well, our project is a lot smaller than PyPy, but still around 20k 
production lines and growing.

We have 2300 tests (unit and functional), and they take one and a half 
hours to run. Over 2/3 of this time is functional tests.

We are just switching over to say that we only need to run unit tests 
before checking in.

What I'm saying certainly seems to be 'very extreme' for most Pythoneers 
that I've talked to (and those on the list here) - but it really is 
pretty standard XP, and the sort of stuff being explored by the agile 
crowd. I have to say that it works *very* well for us, and provides a 
great environment to develop in.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

> Laura Creighton
>
>