[TIP] The test suite that was too dangerous to live
John J Lee
jjl at pobox.com
Fri Oct 23 13:07:07 PDT 2009
On Sat, 26 Sep 2009, Ben Finney wrote:
> I'm working on a project (‘python-daemon’) whose tests involve a lot of
> situational test case setup: inter-process relations, race conditions,
> polymorphic classes, non-deterministic behaviours. With the help of mock
> objects (made *very* simple by the beautiful ‘MiniMock’ library), I'm
> able to tame these environmental hazards, set up a fake environment, and
> assert the behaviour my code should have on its environment.
[...]
> But the strange test
> suite behaviour I saw when making the modifications means I no longer
> have the confidence that my tests are *actually* testing what I *think*
> they're testing.
[...]
First, as Michal said, while I can see that "test doubles" (Meszaros'
terminology) will be useful when dealing with multiple processes, that
doesn't necessarily mean verifying method calls ("mocking"):
http://martinfowler.com/articles/mocksArentStubs.html
A nice example of a test suite in a related domain to yours that leans
more towards functional testing (and away from unit testing) and towards
what Martin Fowler calls "classical TDD" (and away from "mockist TDD"):
http://github.com/mseaborn/coconut-shell
http://plash.beasts.org/wiki/CoconutShell
Also: Dependency injection is good, but don't abandon monkey-patching,
either (I see your scaffold module uses this). It's usefulness does
outweigh the limited obscurity & fragility it introduces, when the code
isn't structured in a way that lends itself to the "classical TDD" style
of testing, so I don't think you should throw the monkey baby out with the
mockist bathwater. A blog post of Mark Seaborn's discusses testing using
monkey-patching, with useful code:
http://lackingrhoticity.blogspot.com/2008/12/helper-for-monkey-patching-in-tests.html
(Mark is also the author of coconut shell, but the tests for that don't
demonstrate use of monkey-patching, as far as I know)
John
More information about the testing-in-python
mailing list