ned at nedbatchelder.com
Wed Apr 22 06:45:11 PDT 2009
I'm coming late to this, but it seems like a classic dilemma. I've run
into the same problem: as my testing efforts intensify, my tests become
more complex. The natural impulse is to factor common complexity out of
the tests into something else. That other thing is the test framework.
Ideally, I'd "refactor" it all the way into someone else's code (for
example, nose). But depending on the project and its audience and
distribution, I may not want to require another dependency, so I have to
make the tradeoff between maintaining my own framework (however slight),
or requiring a third-party package with all of those headaches (tracking
changes, helping my customers install someone else's package, etc).
> > But don't you then have to debug your test framework?
> Your test framework should have unit tests - and yes, you'll need to
> treat it like any other piece of software, tests, docs, and debugging.
I personally haven't ever gotten to the point of needing separate tests
just for the test scaffolding. Generally, the product tests themselves
will alert me to failures in the test runner. I can see a large enough
project getting to that point, though: there is the risk that your test
scaffold will break in a way that means all tests pass without actually
> I'd rather it be simple. No complexity allowed. Didn't you read my blog post??
I'm assuming this is hyperbole for dramatic effect. Of course we all
accept (and even create) complexity. The trick is to find the
Goldilocks point: not too little, not too much, just the right amount.
And then to wrap it in abstractions that don't leak too much.
> A lot of this sounds like something I'd use nose for, but we're trying to keep things stdlib-ish.
Maybe this is the point to examine more closely. The best way to get
rid of complexity is to outsource it to someone else. Then you don't
have to maintain it, but you still get to benefit from its power. What
is it that is keeping you from requiring nose? Is it sacrilege to say
that it's ok to require 3rd-party stuff to run tests, since not all
users of the product will need to run the tests?
Ned Batchelder, http://nedbatchelder.com
More information about the testing-in-python