[TIP] Generating tests, II (was: Re: Meta-test methods...)

Douglas Philips dgou at mac.com
Fri Apr 24 12:02:04 PDT 2009


On or about 2009 Apr 24, at 2:32 PM, C. Titus Brown indited:
> On Fri, Apr 24, 2009 at 12:01:35AM -0400, Douglas Philips wrote:
> -> I said that checking a test method
> -> for being a generator was a hack, seeminly cool at first, but upon
> -> further thought it is just as a proof of concept that it is  
> useful to
> -> have a way to dynamically generate a sequence of test methods/ 
> functions.
> -> Testing only for generators is both opaque and, critically, fails  
> to
> -> generalize to arbitrary python sequences.
>
> This might be a discussion better carried out on nose-dev, but:

Well, maybe, but I want this kind of functionality in my almost- 
vanilla-unittest framework. :)


> isn't one issue that generators are the only "iterator-like"  
> function to
> have a clear signature at the time of definition?  Specifically, how
> else can I know that a test_* function was a test generator rather  
> than
> a function executing a single test?

This is precisely why I suggested using a decorator. You shot that  
down as not being 2.3 compatible. Fair enough. The existing unittest  
framework uses a naming convention (setUp, tearDown, test...) to  
identify the methods it should call and the semantics it expects from  
them. Using the generator-ness of a test method to treat it  
differently is overloading the testing mechanisms in a bad way. (more  
below.)


> 'yield' makes the function have a generator type, which is kind of  
> cool
> to realize -- I hadn't thought of yield as being an implementation of
> function typing in Python before ;)

Not until 2.6 (IIRC) is there a defined way of asking that question.  
nose and py.test both have code that grunge around to determine that  
in other environments (IronPython, older versions of python, etc).


> Anyway, I can't think of a way to write a wrapper that would be able  
> to
> make that distinction for your code below without actually running the
> function and trying to iterate over what it returned.  That might be  
> the
> reason why nose does things the way it does.

I can. Use a decorator to declare your intent (Explicit is better than  
implicit). However I think it is also reasonable to have a separate  
naming convention. When I go to read some test code, the existing  
naming convention is a great help to understanding it. I know what  
setUp is about and when it is call, same for tearDown, and same for  
test... methods. Except, well, in this one case, a test... method  
means something very different and at a different meta-level.

Even if 'isgenerator' is nice and cleanly abstracted away, it still  
wrong to use it:
     Whether a particular meta-test-sequence-returning-function  
(MTSRF) returns a list, tuple, other sequence-supporting type, or a  
generator, is an implementation detail that should not change how the  
result is used or even if it is used.

Sure its a cute hack to use generators that way, but it would be just  
as simple to use a unique prefix. Then, if you find a generator is the  
best way to implement a particular MTSRF, you are free to do so, and  
if returning a tuple is clearer, you don't have to worry that your  
implementation decisions are going to have unintended side-effects.

		-Doug




More information about the testing-in-python mailing list