[TIP] Reporting assertions as a way for test parametrization

[Konrad Delong]

Here's my idea on how to achieve parametrized tests. Let's not
parametrize tests (:>), but instead make assertions report their
results. Example is below. The code lives here
(http://code.google.com/p/konryd-scripts/source/browse/trunk/reporting_assertion/).

This approach has few drawbacks. You can't collect assertions when
doing collect-only run. It's tightly coupled with TestCase
implementation (needs to know how to report results) and - for the
same reason - it works only for assertions bound to TestCase instance
(the decorator has an optional _self= keyword argument for decorating
unbound assertions, but the results are not reusable across TestCase
instances). The last problem is __str__ evaluation: if the objects
under have side-effects in this method, you might get quite unexpected
results.

It is, however quite powerful: the data fed into the assertions is
generated in the code itself (so the parameters don't need to be
static). It's also very easy to use: just decorate your assertion in
order to get the reports.

Run this:

#################################################################################
import unittest
from reporting_assertion import TestCase, reporting

class MyTestCase(TestCase):
    @reporting
    def some_assertion(self, a, b):
        assert a == b
    def test_example(self):
        for x in (1, 2, 3):
            self.some_assertion(x, 2)

if __name__ == "__main__":
    unittest.main()
#################################################################################

In order to get this report:

#################################################################################
$ python example.py
FF.
======================================================================
FAIL: test_example.some_assertion(1, 2) (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/[path]/reporting_assertion/reporting_assertion.py", line 12, in wrapper
    ret = assertion(self, *args, **kwargs)
  File "example.py", line 8, in some_assertion
    assert a == b
AssertionError

======================================================================
FAIL: test_example.some_assertion(3, 2) (__main__.MyTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/[path]/reporting_assertion/reporting_assertion.py", line 12, in wrapper
    ret = assertion(self, *args, **kwargs)
  File "example.py", line 8, in some_assertion
    assert a == b
AssertionError

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (failures=2)
#################################################################################

I described it here:
http://konryd.blogspot.com/2009/05/reporting-assertions-as-way-for-test.html

Konrad