[TIP] Generating tests, II (was: Re: Meta-test methods...)

[C. Titus Brown]

On Fri, Apr 24, 2009 at 03:02:04PM -0400, Douglas Philips wrote:
-> On or about 2009 Apr 24, at 2:32 PM, C. Titus Brown indited:
-> > On Fri, Apr 24, 2009 at 12:01:35AM -0400, Douglas Philips wrote:
-> > -> I said that checking a test method
-> > -> for being a generator was a hack, seeminly cool at first, but upon
-> > -> further thought it is just as a proof of concept that it is  
-> > useful to
-> > -> have a way to dynamically generate a sequence of test methods/ 
-> > functions.
-> > -> Testing only for generators is both opaque and, critically, fails  
-> > to
-> > -> generalize to arbitrary python sequences.
-> >
-> > This might be a discussion better carried out on nose-dev, but:
-> 
-> Well, maybe, but I want this kind of functionality in my almost- 
-> vanilla-unittest framework. :)
-> 
-> > isn't one issue that generators are the only "iterator-like"  
-> > function to
-> > have a clear signature at the time of definition?  Specifically, how
-> > else can I know that a test_* function was a test generator rather  
-> > than
-> > a function executing a single test?
-> 
-> This is precisely why I suggested using a decorator. You shot that  
-> down as not being 2.3 compatible. Fair enough. The existing unittest  
-> framework uses a naming convention (setUp, tearDown, test...) to  
-> identify the methods it should call and the semantics it expects from  
-> them. Using the generator-ness of a test method to treat it  
-> differently is overloading the testing mechanisms in a bad way. (more  
-> below.)

[ ... ]

-> I can. Use a decorator to declare your intent (Explicit is better than  
-> implicit). However I think it is also reasonable to have a separate  
-> naming convention. When I go to read some test code, the existing  
-> naming convention is a great help to understanding it. I know what  
-> setUp is about and when it is call, same for tearDown, and same for  
-> test... methods. Except, well, in this one case, a test... method  
-> means something very different and at a different meta-level.

Hmm, I see now.

First, I'll say that I'm pretty happy with the nose test generator
system as-is and I don't know of any situations where it's actually
caused problems, but I see more clearly now why you dislike it.

Decorators are a good way of signalling that a function is a test
generator, if you aren't forced to use py2.3.  I'm trying to figure out
why I dislike the non-decorator tagging so much; I think because I want
to know what the signature of a test function is up front, without
looking at the end.  That conflicts with 'yield' too, though.  I guess
I'm just inconsistent.

Bottom line, for now, is that when I'm using nose, I don't run into
problems and (especially with the well-defined plug-in interface for
test discovery and execution) I have a specific place to put magic
behavior.  When I'm stuck with unittest, I have a lot of extra
boilerplate code and I don't have a well-defined place to extend things,
both of which frustrate me.  So I'll stick with nose when I can, until
it starts giving me problems...

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu