[TIP] dynamically create tests with unittest

[Albert-Jan Roskam]

----- Original Message -----

> From: Ben Finney <ben+python at benfinney.id.au>
> To: testing-in-python at lists.idyll.org
> Cc: 
> Sent: Thursday, February 26, 2015 1:08 AM
> Subject: Re: [TIP] dynamically create tests with unittest
> 
> Albert-Jan Roskam <fomcl at yahoo.com> writes:
> 
>>  With nose one can use generators to dynamically create tests:
>> 
>>  What is the preferred way to do this with unittest?
> 
> I recommend using the ‘testscenarios’ library
> <URL:https://pypi.python.org/pypi/testscenarios/>:
> 
>     import unittest
>     import testscenarios
> 
>     import system_under_test as sut
> 
>     class foo_func_TestCase(unittest.TestCase, testscenarios.WithScenarios):
>         """ Test cases for the `foo_func` function. 
> """
> 
>         scenarios = [
>                 ('empty baz', {
>                     'baz': [],
>                     'expected_result': "spam",
>                     }),
>                 ('seven baz items', {
>                     'baz': [3, 5, 7, 11, 13, 17, 19],
>                     'expected_result': "beans",
>                     }),
>                 ('default no baz', {
>                     'expected_result': "eggs",
>                     }),
>                 ]
> 
>         def test_returns_expected_result_given_baz(self):
>             """ Should return the expected result given the `baz` 
> value. """
>             baz = getattr(self, 'baz', None)
>             result = sut.foo_func(baz)
>             self.assertEqual(self.expected_result, result)
> 
>         def test_quux_remains_unchanged(self):
>             """ Should not affect the `quuz` value. 
> """
>             quux_prev = sut.quux
>             baz = getattr(self, 'baz', None)
>             sut.foo_func(baz)
>             self.assertEqual(quux_prev, sut.quux)
> 
> That TestCase class will cause six text cases to be generated during the
> test run: one for each combination of scenario and ‘test_*’ function.
> The TestRunner output for each test case will include both the function
> name and the scenario name, to make it clear which condition caused any
> failure.
> 
> Each test case, when created dynamically, will have attributes as
> specified by the name → value mapping for the scenario. Those attributes
> can be used however the test case function chooses.
> 
> Adding new scenarios is then a simple matter of adding new entries in
> the scenario collection for the class; all the test case methods will be
> run with all the scenarios, as distinct test cases in the run.


Hi all,

Thanks very much for your responses.


I am fetching records from a database, and based on metadata and other parameters these records are formatted. For example,  the record would contain floats that represent the number of seconds since the gregorian epoch. I currently test with assertEqual(desired_record, actual_record). Instead of making comparisons by *record*, I would like to generate one test for each *value* to be compared. The following is probably wrong as it is much like writing manually writing the tests, but it illustrates (I hope) what I want to do:

scenarios = [
                ('european date', {
                    'gregorian': 11654150400.0,
                    'fmt': '%d-%m-%Y',  
                    'as_ustring': False, 
                    'expected_result': b'03-02-1952,
                    }),
                ('european datetime, pre 1900', {
                    'gregorian': 0.0,
                    'fmt': '%Y-%m-%d %H:%M:%S',  
                    'as_ustring': False, 
                    'expected_result': b'1582-10-14 00:00:00',
                    }),
                ('american date', {
                    'gregorian': 11654150400.0,

                    'fmt': '%m-%d-%Y',
                    'as_ustring': False,                     'expected_result': b'02-03-1952',
                    }),
                ('iso date', {
                    'gregorian': 11654150400.0,

                                                          'fmt': '%Y-%m-%d', 
                    'as_ustring': False,
                    'expected_result': b'1952-02-03',
                    }),
                ]

I would have expected to have something like (pseudo code):

for desired_value, actual_value, label in zip(desired_record, actual_record, labels):
     generate_test(desired_value, actual_value, label)

Something like that. I hope I am not too vague now. :-)

Regards,
Albert-Jan