[TIP] Functional Testing of Desktop Applications

[holger krekel]

On Sun, Mar 04, 2007 at 17:49 -0800, Grig Gheorghiu wrote:
> --- Laura Creighton <lac at openend.se> wrote:
> 
> > In a message of Mon, 05 Mar 2007 00:01:35 GMT, Michael Foord writes:
> > <snip>
> > >This leads us into a bit of a debate about how much to mock out in
> > our 
> > >unit tests. The most extreme testing doctrine says that you should
> > mock 
> > >out *all* your dependencies when testing a unit - even dependencies 
> > >within the same object (if you are testing method 'a' and it
> > contains a 
> > >call to function or method 'b' you should mock 'b').
> > >
> > 
> > A different, related problem.  The idea is that your tests stay live
> > and you run them all periodically to make sure nothing breaks.  Do
> > this properly, and write a thing like pypy (and py.test) and you
> > will find that a run of all your tests takes 3-4 hours. :-(  Thus
> > the most extreme doctrine above only works on "small-enough"
> > projects.
> 
> I think that mocking actually speeds things up tremendously, while also
> complicating them....But for a project like PyPy, which does lots of
> backend stuff, I would think mocking would be beneficial in many cases
> -- so that functionality gets tested in isolation. It would be
> interesting to know whether the PyPy guys actually do any mocking in
> their testing.

yes we do but it's not a dominant approach.  
I guess PyPy could be stricter regarding reviewing tests, 
we do have a lot more functional ones than i'd wish sometimes.  

In a related note, we are going for ad-hoc distributed testing, 
using several machines at once to distribute from a local 
non-committed working copy, we also start to have nice 
ways to distribute from unix to windows nicely :) 

For a demo, you might check out the current py.test release 
(http://codespeak.net/py) and issue "py.test --dist -w" 
on your local working copy, it will tell you to have a
conftest.py with "dist_hosts = ['hostname1', 'hostname2', ...]"
ssh hosts and then fire up a webserver reporting the arriving test
events from all these hosts, interactively inspecting failures. 
(the web app is generated by PyPy of course :) 

best, 

holger