[TIP] Generating tests, II (was: Re: Meta-test methods...)
C. Titus Brown
ctb at msu.edu
Fri Apr 24 12:10:22 PDT 2009
On Fri, Apr 24, 2009 at 03:02:04PM -0400, Douglas Philips wrote:
-> On or about 2009 Apr 24, at 2:32 PM, C. Titus Brown indited:
-> > On Fri, Apr 24, 2009 at 12:01:35AM -0400, Douglas Philips wrote:
-> > -> I said that checking a test method
-> > -> for being a generator was a hack, seeminly cool at first, but upon
-> > -> further thought it is just as a proof of concept that it is
-> > useful to
-> > -> have a way to dynamically generate a sequence of test methods/
-> > functions.
-> > -> Testing only for generators is both opaque and, critically, fails
-> > to
-> > -> generalize to arbitrary python sequences.
-> > This might be a discussion better carried out on nose-dev, but:
-> Well, maybe, but I want this kind of functionality in my almost-
-> vanilla-unittest framework. :)
-> > isn't one issue that generators are the only "iterator-like"
-> > function to
-> > have a clear signature at the time of definition? Specifically, how
-> > else can I know that a test_* function was a test generator rather
-> > than
-> > a function executing a single test?
-> This is precisely why I suggested using a decorator. You shot that
-> down as not being 2.3 compatible. Fair enough. The existing unittest
-> framework uses a naming convention (setUp, tearDown, test...) to
-> identify the methods it should call and the semantics it expects from
-> them. Using the generator-ness of a test method to treat it
-> differently is overloading the testing mechanisms in a bad way. (more
[ ... ]
-> I can. Use a decorator to declare your intent (Explicit is better than
-> implicit). However I think it is also reasonable to have a separate
-> naming convention. When I go to read some test code, the existing
-> naming convention is a great help to understanding it. I know what
-> setUp is about and when it is call, same for tearDown, and same for
-> test... methods. Except, well, in this one case, a test... method
-> means something very different and at a different meta-level.
Hmm, I see now.
First, I'll say that I'm pretty happy with the nose test generator
system as-is and I don't know of any situations where it's actually
caused problems, but I see more clearly now why you dislike it.
Decorators are a good way of signalling that a function is a test
generator, if you aren't forced to use py2.3. I'm trying to figure out
why I dislike the non-decorator tagging so much; I think because I want
to know what the signature of a test function is up front, without
looking at the end. That conflicts with 'yield' too, though. I guess
I'm just inconsistent.
Bottom line, for now, is that when I'm using nose, I don't run into
problems and (especially with the well-defined plug-in interface for
test discovery and execution) I have a specific place to put magic
behavior. When I'm stuck with unittest, I have a lot of extra
boilerplate code and I don't have a well-defined place to extend things,
both of which frustrate me. So I'll stick with nose when I can, until
it starts giving me problems...
C. Titus Brown, ctb at msu.edu
More information about the testing-in-python