[TIP] Functional Testing of Desktop Applications

[Vladimir Ignatov]

Hello,

> >> We have 2300 tests (unit and functional), and they take one and a half
> >> hours to run. Over 2/3 of this time is functional tests.
> For the functional tests we don't mock *anything* (well... except the
> blocking modal system dialogs). A lot of these tests hit the filesystem,
> the network and databases. We want our functional tests to test *actual*
> behaviour, not behaviour in the presence of mocks.
> Each functional test launches the application, so we have some startup
> time overhead per test.
> Some of these tests involve checking behaviour when the network is
> disconnected, attempting to connect to non-existent servers etc - so
> there are delays waiting for timeouts.

Well, that sounds cool. But...
We should understand that this is _not_ a classical TDD that described
in the brilliant "Test Driven Development: By Example" book. It is not
good nor bad - it is just a fact.

Your approach actually very close to Kent's example described in the
"Isolated test" chapter:

Here is the citate:
"When I was a young programmer—long, long ago when we had to dig our
own bits out of the snow and carry heavy buckets of them barefooted
back to our cubicles, leaving bloody little footprints for the wolves
to follow… . Sorry, just reminiscing. My first experience of automated
tests was having a set of long-running, overnight, GUI-based tests
(you know, record the keystrokes and mouse events and play them back)
for a debugger I was working on. (Hi Jothy, hi John!) Every morning
when I came in, there would be a neat stack of paper on my chair
describing last night's test runs. (Hi Al!) On good days there would
be a single sheet summarizing that nothing broke. On bad days there
would be many, many sheets, one for each broken test. I began to dread
days when I saw a pile of paper on my chair."

Long running tests smells because they broke the main idea of TDD -
you will not use such tests for leading your __development__.  And
since TDD is about the development and not about testing this is fatal
for the whole idea. Just using tests not mean using TDD. And that is
you case. Once again I don't say this is bad. I just say the fact it
is not TDD. There is a yahoo group about extreme programming where
such guys as Kent Beck or Ron Jeffries subscribed. You can asks them
to comment you approach if you like.

Vladimir Ignatov