[TIP] hackish generative tests (was: Re: Generating tests, II (was: Re: Meta-test methods...))
holger at merlinux.eu
Tue Apr 28 04:17:47 PDT 2009
i guess i am somewhat responsible for the idea of "generative
tests" and maybe you find it suprising to hear: i mostly
agree with what you say. Tests generators as they stand
are somewhat hackish. Some more inline comments ...
On Fri, Apr 24, 2009 at 15:35 -0400, Douglas Philips wrote:
> On or about 2009 Apr 24, at 3:10 PM, C. Titus Brown indited:
> > I'm trying to figure out why I dislike the non-decorator tagging so
> > much; I think because I want
> > to know what the signature of a test function is up front, without
> > looking at the end. That conflicts with 'yield' too, though. I guess
> > I'm just inconsistent.
> Cracking open a method and examining how it is implemented is just
> plain and simple: a hack. If it were in code I was reviewing, I would
> classify it as a bug. If there were no other way, sure, it might be
> tolerable. But there are other ways. At least two that that are
> actually Pythonic.
"cracking open a method" i don't consider a fitting term
for checking if a function is a generator.
In any case, introspection and reflection is something i consider pythonic
so if you want to say that using such is a hack in general, i disagree.
Testing is an application of meta-programming and meta-programming
naturally involves reflection techniques.
> If the py.test/nose dudes had used new naming convention, would you
> really have balked at it? Would you have thought it complicated or un-
> usable? Would you have thought "Oh, that makes sense, this is a
> different part of the testing eco-system" of course it needs a
> different name?
... this is a good point. Dual-using the "test_*" naming
convention for test generators i now consider a bad idea that
lead to implementation complications both for py.test and nose.
In that sense i agree to your "cracking open a method" critcism.
FYI i also agree to some of the critical viewpoints on generative tests
However, I think that instrumenting a test run to invoke tests
with multiple different parameters or in multiple environments
is useful and worthwhile.
So i'd like to phase out (probably making it an
optional plugin) yield-mediated generative tests once
i implemented a mechanism based on
that allows to invoke a test function repeatedly
with multiple parameter sets.
More information about the testing-in-python