[TIP] Generating tests, II (was: Re: Meta-test methods...)
Douglas Philips
dgou at mac.com
Fri Apr 24 12:02:04 PDT 2009
On or about 2009 Apr 24, at 2:32 PM, C. Titus Brown indited:
> On Fri, Apr 24, 2009 at 12:01:35AM -0400, Douglas Philips wrote:
> -> I said that checking a test method
> -> for being a generator was a hack, seeminly cool at first, but upon
> -> further thought it is just as a proof of concept that it is
> useful to
> -> have a way to dynamically generate a sequence of test methods/
> functions.
> -> Testing only for generators is both opaque and, critically, fails
> to
> -> generalize to arbitrary python sequences.
>
> This might be a discussion better carried out on nose-dev, but:
Well, maybe, but I want this kind of functionality in my almost-
vanilla-unittest framework. :)
> isn't one issue that generators are the only "iterator-like"
> function to
> have a clear signature at the time of definition? Specifically, how
> else can I know that a test_* function was a test generator rather
> than
> a function executing a single test?
This is precisely why I suggested using a decorator. You shot that
down as not being 2.3 compatible. Fair enough. The existing unittest
framework uses a naming convention (setUp, tearDown, test...) to
identify the methods it should call and the semantics it expects from
them. Using the generator-ness of a test method to treat it
differently is overloading the testing mechanisms in a bad way. (more
below.)
> 'yield' makes the function have a generator type, which is kind of
> cool
> to realize -- I hadn't thought of yield as being an implementation of
> function typing in Python before ;)
Not until 2.6 (IIRC) is there a defined way of asking that question.
nose and py.test both have code that grunge around to determine that
in other environments (IronPython, older versions of python, etc).
> Anyway, I can't think of a way to write a wrapper that would be able
> to
> make that distinction for your code below without actually running the
> function and trying to iterate over what it returned. That might be
> the
> reason why nose does things the way it does.
I can. Use a decorator to declare your intent (Explicit is better than
implicit). However I think it is also reasonable to have a separate
naming convention. When I go to read some test code, the existing
naming convention is a great help to understanding it. I know what
setUp is about and when it is call, same for tearDown, and same for
test... methods. Except, well, in this one case, a test... method
means something very different and at a different meta-level.
Even if 'isgenerator' is nice and cleanly abstracted away, it still
wrong to use it:
Whether a particular meta-test-sequence-returning-function
(MTSRF) returns a list, tuple, other sequence-supporting type, or a
generator, is an implementation detail that should not change how the
result is used or even if it is used.
Sure its a cute hack to use generators that way, but it would be just
as simple to use a unique prefix. Then, if you find a generator is the
best way to implement a particular MTSRF, you are free to do so, and
if returning a tuple is clearer, you don't have to worry that your
implementation decisions are going to have unintended side-effects.
-Doug
More information about the testing-in-python
mailing list