[TIP] paramterized or generative tests [was nose2: the nosening]

[Rob Guttman]

Here's the syntax and functionality I prefer:

  @permute( x=[1, 2], y=[0, 1, 3] )
  def test_addition(self, x, y):
    self.assertEqual( x + y, 2 )

... resulting in *six* tests (one for each permuted combination) and of
these two passes and four fails.  In my test tool, each of the six tests are
tagged with each param and value so nose's attrib plugin can select them:

  test_addition.x=1
  test_addition.y=0

  test_addition_2.x=1
  test_addition_2.y=1

  test_addition_3.x=1
  test_addition_3.y=3

  ...

This permuting helps us write a single test for an API method, for example,
that spans multiple versions and environments (plus other params).  The
current way that the tool works, the new tests are created and added to the
test class on module import - so from unittest's perspective they look like
normal test methods.  In other words, nose doesn't need to do anything
special to deal with them and I suspect unittest2 won't either.  That said,
I would prefer not to do the heavy work of permuting tests on module import,
so if there's a better way to do this, I would be interested in hearing
about it.

Thanks.
- Rob


On Thu, Aug 5, 2010 at 10:20 PM, Brian Curtin <brian.curtin at gmail.com>wrote:

> On Thu, Aug 5, 2010 at 20:56, Michael Foord <fuzzyman at voidspace.org.uk>wrote:
>
>> On 06/08/2010 02:49, Jesse Noller wrote:
>>
>>> On Thu, Aug 5, 2010 at 9:31 PM, Michael Foord<fuzzyman at voidspace.org.uk>
>>>  wrote:
>>>
>>>
>>>> On 06/08/2010 02:23, Jesse Noller wrote:
>>>>
>>>>
>>>>> [snip...]
>>>>> ps: If nose2/unittest2 could find it in it's shiny new innocent heart
>>>>> to say, have support for test generators; I'd die happy.
>>>>>
>>>>>
>>>>>
>>>> Well there's another discussion :-)
>>>>
>>>> It would be very simple to support parameterized tests (of some form),
>>>> where
>>>> the number of tests can be known at test load time. Supporting
>>>> generative
>>>> tests - where the tests are generated at *test run time* is uglier.
>>>>
>>>> Would parameterized tests, where the tests are all generated at test
>>>> load
>>>> time, suffice or do you have a use case for generative tests where the
>>>> tests
>>>> have to be generated whilst the tests are running and not when the tests
>>>> are
>>>> loaded?
>>>>
>>>> Michael
>>>>
>>>>
>>> For me? To be honest, 98% of my (and the people around me) use cases
>>> are the parameterized case - the test generator syntax is just pure
>>> awesome (Ok, so I really like it) tests generated at run time tends to
>>> rub most people wrong :)
>>>
>>>
>>
>> Ok. Good syntax is generally a good thing, and there is no reason why a
>> generator couldn't be called at test load time to generate the tests -
>> perhaps in conjunction with a decorator so we don't have to do horrible
>> introspection to determine if something is a test generator or not.
>>
>> Could you give me an example usage please, so I can think about how to
>> support it (probably initially as a plugin).
>>
>> All the best,
>>
>> Michael
>
>
> I've wanted to do something like this in the past...
>
> @unittest.params((1, 1), (2, 0),  (1, 2))
> def test_addition(self, x, y):
>     self.assertEqual(x + y, 2)
>
> ...resulting in two passes, one fail.
>
> I think I already mentioned this or something like it on the Roundup issue
> for this topic. I have part of a patch around somewhere, but it's far from
> complete.
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20100805/13c95991/attachment.htm>