[TIP] Meta-test methods...

[Douglas Philips]

On or about 2009 Apr 27, at 7:40 PM, Robert Collins indited:
> On Mon, 2009-04-27 at 13:39 -0400, Doug Philips wrote:
>> I can see how to use that when the scenario lists can be computed  
>> at module load time, I'm still not sure I get how to compute them  
>> "late" (as per my examples in the previous emails) when the precise  
>> "values" are not known until the test method is run...
...
> if __name__ == '__main__':
>    # can't use TextTestRunner as we don't have a load hook available
>    # in python core yet, so we need a bit of plumbing that the test
>    # runner normally does.
>    loader = TestLoader()
>    tests = loader.loadTestsFromName(__name__)

>    # arbitrary delay goes here

>    # Now we decide what scenarios we want
>    DemoTest.scenarios.append(('01', dict(start=0, end=1)))
>    DemoTest.scenarios.append(('10', dict(start=1, end=0)))
>    # Now we're actually about to run.
>    suite = TestSuite(generate_scenarios(tests))
>    print suite.countTestCases()
>    for test in suite:
>        print test.id()
>    suite.run(_TextTestResult(_WritelnDecorator(sys.stdout), False, 2))

I feared that was where you would be going.
I really don't want to pull the logic out of my test files and have to  
put it somewhere else just to generate scenarios.
I think, though, that if I added another generic "pass", I could scan  
all the loaded test modules/classes for "set up your scenarious" code.  
But I'll have to think about it. The problem I have just jumping on  
this bandwagon is that its not a no-brainer for our existing structure  
(which may say as much about us as it does about your technique).  
Specifically, we provide environmental information (device  
personality) orthogonally to the test code (table driven, in essence).  
I really don't want to have to get into 'which piece is loaded first/ 
last' kinds of fragility. Right now all that matters is that by the  
time the test method(s) execute, all the pieces have been loaded. I'll  
have to think about this and the funcargs stuff more deeply.
Its an itch I need to scratch right now :(, so I'm probably going to  
have to go with some hybrid and simplify once I've gotten some more  
experience.

-Doug