[TIP] Unittest Changes

[Andrew Bennetts]

Michael Foord wrote:
[...]
> 
> Can you suggest how it might be implemented in unittest?
> 
> If it can be done *cleanly* then it shouldn't be too controversial.
> 
> If a  module has this attribute:
> 
>     test_suite = 'something.else'
> 
> Then how should unittest invoke 'something.else' ? (And must 
> 'something.else' then present a suite compatible with a unittest 
> TestRunner ?)

It's worth pointing out that at least Twisted's test runner will look
for and execute “test_suite” callables in test modules.   If a test_suite
callable is present, it will call it to construct the test suite rather than
doing the default loadTestsFromModule stuff.  The return value of the
test_suite callable is expected to be a TestSuite.

Bazaar's test runner does something similar, except it looks for a “load_tests”
callable with a signature of (standard_tests, module, loader).  The docstring on
the extended loadTestsFromModule says in part:

    “load_tests should be defined like so:
    >>> def load_tests(standard_tests, module, loader):
    >>>    pass

    standard_tests is the tests found by the stock TestLoader in the
    module, module and loader are the module and loader instances.

    For instance, to run every test twice, you might do:
    >>> def load_tests(standard_tests, module, loader):
    >>>     result = loader.suiteClass()
    >>>     for test in iter_suite_tests(standard_tests):
    >>>         result.addTests([test, test])
    >>>     return result
    ”

Bazaar's test suite mainly uses this hook to build parameterised TestCases, i.e.
to run all the transport_implementations tests against all the Transport
subclasses.  See the load_tests function in
<http://bazaar.launchpad.net/~bzr/bzr/trunk/annotate/head:/bzrlib/tests/test_transport_implementations.py>
for a real example.  An advantage of load_tests over test_suite AIUI is that it
allows bzrlib's test loader to filter test cases before invoking load_tests
(e.g. with a regex passed on the “bzr selftest” command line), which can save
significant time if you are running just a subset of the full suite.  With
test_suite() you have no choice but to filter afterwards, by which time you may
have instantiated hundreds of test case instances that you'll never run.

Ideally one of these existing protocols would become standardised. :)

-Andrew.