[TIP] paramterized or generative tests [was nose2: the nosening]

Michael Foord fuzzyman at voidspace.org.uk
Thu Aug 5 19:10:43 PDT 2010

On 06/08/2010 03:05, Jesse Noller wrote:
> On Thu, Aug 5, 2010 at 9:56 PM, Michael Foord<fuzzyman at voidspace.org.uk>  wrote:
>> On 06/08/2010 02:49, Jesse Noller wrote:
>>> On Thu, Aug 5, 2010 at 9:31 PM, Michael Foord<fuzzyman at voidspace.org.uk>
>>>   wrote:
>>>> On 06/08/2010 02:23, Jesse Noller wrote:
>>>>> [snip...]
>>>>> ps: If nose2/unittest2 could find it in it's shiny new innocent heart
>>>>> to say, have support for test generators; I'd die happy.
>>>> Well there's another discussion :-)
>>>> It would be very simple to support parameterized tests (of some form),
>>>> where
>>>> the number of tests can be known at test load time. Supporting generative
>>>> tests - where the tests are generated at *test run time* is uglier.
>>>> Would parameterized tests, where the tests are all generated at test load
>>>> time, suffice or do you have a use case for generative tests where the
>>>> tests
>>>> have to be generated whilst the tests are running and not when the tests
>>>> are
>>>> loaded?
>>>> Michael
>>> For me? To be honest, 98% of my (and the people around me) use cases
>>> are the parameterized case - the test generator syntax is just pure
>>> awesome (Ok, so I really like it) tests generated at run time tends to
>>> rub most people wrong :)
>> Ok. Good syntax is generally a good thing, and there is no reason why a
>> generator couldn't be called at test load time to generate the tests -
>> perhaps in conjunction with a decorator so we don't have to do horrible
>> introspection to determine if something is a test generator or not.
>> Could you give me an example usage please, so I can think about how to
>> support it (probably initially as a plugin).
>> All the best,
> Pretty simple load test use case - this could be a generative test,
> but I tend to favor outlining it explicitly most of the time:
>      def test_varied_writes(self):
>          """ Perform write tests with varying threads and sizes - sizes
>          are in megabytes. ent format = (size, num threads)
>          """
>          for ent in ((5,1), (5,5), (5,10), (10,1), (10,5), (10,10))
>              yield self._run_load, ent
> Really, this is the most simple case. You iterate / enumerate some
> *thing* (could be a dict of configurations for the system under test)
> and just dispatch a shared method, passing in the args.

Should be easy to support this by exhausting the generator and creating 
the tests at test load time, using a decorator to mark the function / 
method as a generator test. If the test is a method on a testcase we do 
it by instantiating the test case in the usual way, but wrap each test 
in an execution block that will call the testcase setup / teardown / 
cleanups correctly. (Or probably modify TestCase.run to reuse its 
existing machinery for this.)

This doesn't preclude supporting any other form of paramterized test in 
a plugin as well.

Cool - thanks!

> jesse


READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

More information about the testing-in-python mailing list