[TIP] paramterized or generative tests [was nose2: the nosening]

[Jesse Noller]

On Thu, Aug 5, 2010 at 9:56 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> On 06/08/2010 02:49, Jesse Noller wrote:
>>
>> On Thu, Aug 5, 2010 at 9:31 PM, Michael Foord<fuzzyman at voidspace.org.uk>
>>  wrote:
>>
>>>
>>> On 06/08/2010 02:23, Jesse Noller wrote:
>>>
>>>>
>>>> [snip...]
>>>> ps: If nose2/unittest2 could find it in it's shiny new innocent heart
>>>> to say, have support for test generators; I'd die happy.
>>>>
>>>>
>>>
>>> Well there's another discussion :-)
>>>
>>> It would be very simple to support parameterized tests (of some form),
>>> where
>>> the number of tests can be known at test load time. Supporting generative
>>> tests - where the tests are generated at *test run time* is uglier.
>>>
>>> Would parameterized tests, where the tests are all generated at test load
>>> time, suffice or do you have a use case for generative tests where the
>>> tests
>>> have to be generated whilst the tests are running and not when the tests
>>> are
>>> loaded?
>>>
>>> Michael
>>>
>>
>> For me? To be honest, 98% of my (and the people around me) use cases
>> are the parameterized case - the test generator syntax is just pure
>> awesome (Ok, so I really like it) tests generated at run time tends to
>> rub most people wrong :)
>>
>
> Ok. Good syntax is generally a good thing, and there is no reason why a
> generator couldn't be called at test load time to generate the tests -
> perhaps in conjunction with a decorator so we don't have to do horrible
> introspection to determine if something is a test generator or not.
>
> Could you give me an example usage please, so I can think about how to
> support it (probably initially as a plugin).
>
> All the best,

Pretty simple load test use case - this could be a generative test,
but I tend to favor outlining it explicitly most of the time:

    def test_varied_writes(self):
        """ Perform write tests with varying threads and sizes - sizes
        are in megabytes. ent format = (size, num threads)
        """
        for ent in ((5,1), (5,5), (5,10), (10,1), (10,5), (10,10))
            yield self._run_load, ent

Really, this is the most simple case. You iterate / enumerate some
*thing* (could be a dict of configurations for the system under test)
and just dispatch a shared method, passing in the args.

jesse