[TIP] Ideology

Jesse Noller jnoller at gmail.com
Tue Apr 21 08:11:49 PDT 2009


On Tue, Apr 21, 2009 at 10:59 AM, C. Titus Brown <ctb at msu.edu> wrote:
> On Tue, Apr 21, 2009 at 10:40:45AM -0400, Doug Hellmann wrote:
> -> On Apr 21, 2009, at 10:19 AM, Jesse Noller wrote:
> -> >On Apr 21, 2009, at 10:11 AM, "C. Titus Brown" <ctb at msu.edu> wrote:
> -> >
> -> >>Over on pygr-dev, I just came up with this bon mot:
> -> >>
> -> >>"""
> -> >>To paraphrase the regexp saying: you have a problem with your tests,
> -> >>so
> -> >>you make your test framework more complicated.  Now you have TWO
> -> >>problems...
> -> >>"""
> -> >>
> -> >>We're talking about making a number of custom extensions to unittest
> -> >>and
> -> >>modifications to our test runner, and in doing so we're adding
> -> >>significantly to the complexity of our test runner.  That Seems Bad.
> -> >>
> -> >>Have other people run into problems doing this or should I stop
> -> >>worrying
> -> >>and just shut up?  (hah)
> -> >
> -> >Better the complexity lie in the runner an framework than in the
> -> >tests. So long as the execution environment can be recreated and run
> -> >outside of the framework, I would say you're ok.
> -> >
> -> >Tests should be simple and straightforward to write, if that means
> -> >making the executor more complex, well so be it. :)
> ->
> -> I'll second that.  Our tests above the unit level take some fairly
> -> complicated fixtures (database settings, lots of filesystem objects
> -> with peculiar permissions, etc.).  Trying to build all of that into
> -> the test itself lead to a lot of pain, so we moved to separate fixture
> -> classes that are easier to reuse.
> ->
> -> The more you can isolate the ugly parts to make it easy for test
> -> authors, the more (and better) tests you'll have.
>
> But don't you then have to debug your test framework?
>

Your test framework should have unit tests - and yes, you'll need to
treat it like any other piece of software, tests, docs, and debugging.

Better to have to test and debug a framework, then to shove the
complexity down into hundreds of tests, which would ultimately result
in fantastic amounts of copy/pasting within the tests.

It's a matter of choice: where do you want the complexity to lie? In
the framework designer's bucket, or the test designer's? I tend
towards the latter obviously (consider the abstractions that Django
provides).

jesse



More information about the testing-in-python mailing list