[TIP] The test suite that was too dangerous to live

John J Lee jjl at pobox.com
Fri Oct 23 13:07:07 PDT 2009


On Sat, 26 Sep 2009, Ben Finney wrote:
> I'm working on a project (‘python-daemon’) whose tests involve a lot of
> situational test case setup: inter-process relations, race conditions,
> polymorphic classes, non-deterministic behaviours. With the help of mock
> objects (made *very* simple by the beautiful ‘MiniMock’ library), I'm
> able to tame these environmental hazards, set up a fake environment, and
> assert the behaviour my code should have on its environment.
[...]
> But the strange test
> suite behaviour I saw when making the modifications means I no longer
> have the confidence that my tests are *actually* testing what I *think*
> they're testing.
[...]

First, as Michal said, while I can see that "test doubles" (Meszaros' 
terminology) will be useful when dealing with multiple processes, that 
doesn't necessarily mean verifying method calls ("mocking"):

http://martinfowler.com/articles/mocksArentStubs.html


A nice example of a test suite in a related domain to yours that leans 
more towards functional testing (and away from unit testing) and towards 
what Martin Fowler calls "classical TDD" (and away from "mockist TDD"):

http://github.com/mseaborn/coconut-shell
http://plash.beasts.org/wiki/CoconutShell


Also: Dependency injection is good, but don't abandon monkey-patching, 
either (I see your scaffold module uses this).  It's usefulness does 
outweigh the limited obscurity & fragility it introduces, when the code 
isn't structured in a way that lends itself to the "classical TDD" style 
of testing, so I don't think you should throw the monkey baby out with the 
mockist bathwater.  A blog post of Mark Seaborn's discusses testing using 
monkey-patching, with useful code:

http://lackingrhoticity.blogspot.com/2008/12/helper-for-monkey-patching-in-tests.html


(Mark is also the author of coconut shell, but the tests for that don't 
demonstrate use of monkey-patching, as far as I know)


John


More information about the testing-in-python mailing list