[TIP] Functional Testing of Desktop Applications

[Michael Foord]

Grig Gheorghiu wrote:
> --- Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> [snip..]
> Well, people have different names for different types of tests. What
> you call functional tests, other people call system/integration tests.
>
> The way I see it, it's not the names that are important, but the
> different types of testing that need to be done at different levels:
>
> * at the unit level (unit testing)
> * at the application logic level below the GUI (functional, or
> integration, however you want to call it; the idea is that you exercise
> a path through several units of your code) 
> * at the GUI level. 
>
>   
Right.

At Resolver we have good test coverage at the GUI level (what we call 
Functional Tests).

We have excellent 'unit test' coverage, but in fact they are a sporadic 
mixture of the next two levels down.

This leads us into a bit of a debate about how much to mock out in our 
unit tests. The most extreme testing doctrine says that you should mock 
out *all* your dependencies when testing a unit - even dependencies 
within the same object (if you are testing method 'a' and it contains a 
call to function or method 'b' you should mock 'b').

When you have a short function that has several dependencies this can 
make writing the tests a real chore. It *also* means that you don't 
catch errors at a higher level - where you mock you test adequately what 
you *think* the wiring between your units is doing, but your mental 
model may be just as buggy as your code.

I'd be reluctant to add an *extra* level of testing (we already have a 
three-to-one ratio between test and production code). We're moving 
towards a less extreme mocking policy I think though, making sure we 
always have some higher level tests that exercise units together and not 
just in isolation.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml


> It's hard to achieve a good test coverage at the application logic
> level below the GUI, but it's well worth it IMO. As I said, it forces
> you into a cleaner application design.
>
> Grig
>
>