[TIP] hackish generative tests (was: Re: Generating tests, II (was: Re: Meta-test methods...))

[Doug Philips]

On or about Wednesday, May 06, 2009, at 05:12AM, holger krekel indited:
>funcargs are instantiated at setup time for a test function, after collection. 

Does "setup time" means "just before .setUp() is called?"

>do "make_tests" prepare the environment/device/perform setup so that 
>your returned-back ("generated") tests can run in that given setup? 

Well, yes, and no. :)
By the time any "make_tests" prefixed method runs, all that preparation has been done.
Hmmm, I realize I wasn't clear in my description.
What I'm doing is returning a TestCase object, as if it had been created as part of the normal discovery process.
So all the preparation that needs to be done before any tests can be run has been done, but the returned object is treated as any other, so it will have its .setUp run, etc.

>do you have multiple "make_tests" that are specific for a
>test-case / test file? 

For my first conversion, no, but we have some that probably will (TestCase sub-classes with multiple make_tests... methods.)


>I think it makes sense to have some "make_test" like way of
>generating tests - maybe "make_test_something" would be called
>for each "test_something", and be passed the "function" object
>that you can already see in the issue?  At least i rather
>introduce a new naming conventions than reuse the "test_*" naming ;) 

Not sure I completely understand you there, but what i think you said feels a bit too heavy weight for me.
Here is a very -incomplete- rough_sketch of what I've done. (I've elided the boring parts. :) )
Not saying that I'm done or that this is how it should be, of course!

class Scenario(TestCase):
    def __init__(self, p1, p2, p3, ...):
        super(...)
        # process scenario specific options p1, etc.
        self.set_test_method(self.execute_scenario, name=self.construct_scenario_test_name())

    # if needed/appropriate, define setUp and tearDown...

    def execute_scenario(self):
        # ...


class FooTest(TestCase):
    def make_tests_for_common_scenarios(self):
         # Code that computes p1s, p2s, etc. based on device personality data.
         return [Scenario(p1, p2, p3, ...) for p1 in p1s...]

--Doug