[TIP] Test discovery for unittest

[Olemis Lang]

On Mon, Apr 6, 2009 at 9:49 AM, Olemis Lang <olemis at gmail.com> wrote:
> On Fri, Apr 3, 2009 at 9:52 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> Ned Batchelder wrote:
>>> This sounds like what nose does.  Are you taking a different approach?
>>
>> Yes, I want to put it in unittest. :-)
>>
>
> Please, I would like to know something. In XUnit test discovery is
> performed by test loaders. And that's according to the philosophy
> behind that framework. I really dont understand why other frameworks
> have been writing multiple plugin systems and complicated stuff since
> all this was already there since a long time (perhaps needing
> improvements, ok )
>
> I mean this is the reason why I wrote MultiTestLoader and
> PackageTestLoader classes in dutest. If you want to load multiple
> types of tests from a single (module | file), provided that a loader
> does nothing but loading the tests written using a single and
> well-known style (e.g. unittest.TestCase & doctests ;) :
>
> {{{
> #!python
>
> loader1 = dutest.DocTestLoader()
> loader2 = unittest.TestLoader()
> loader3 = TheBestLoaderEver()
>
> l = MultiTestLoader(loader1, loader2, loader3)
>
> l.loadTestsFromModule(mymodule)
>
> }}}
>

Oops !  I forgot to mention : the result of doing this is a test suite
containing the individual suites loaded by loader1, loader2, loader3

This is some kind of chain of responsibility

And all this is provided in dutest module together with loaders for doctests

Besides if you want to load tests from different files you use
PackageTestLoader as follows :

{{{
#!python

ul = unittest.TestLoader()

# Remember the previous example of functional benchmarking?
pl = dutest.PackageTestLoader(pattern='oop.test.exdbc_*', loader=ul)

}}}

The resulting suite contains the tests found in oop.test.exdbc_ (see [3]_)

The best is that you can combine them like this

{{{
#! python

# Put all the snippets together ;)
pl = dutest.PackageTestLoader(pattern='oop.test.exdbc_*', loader=l)
}}}

and in this case you will be loading all doctests, instances of
TestCase, and if you include more loaders in MultiTestLoader well,
more of them; but only those found in (pls see [3]_)
oop.test.exdbc_contract.py, oop.test.exdbc_ipdbc.py, exdbc_oopdbc.py,
exdbc_pydbc.py ...

;o)

I just saw your code and I insist :

- Test suites are not there to discover and load test cases, that's
test loaders' job. I just dont like DiscoveringTestSuite since I dont
think it is correct considering XUnit philosophy ( nothing personal ;)

- Using functions like `find_files` reminds me doctest unittest API
and that kind of code does not scale. Loaders and decorators have been
invented for that. For example, what if my modules are somewhere else
and not directly in the file system ? What if I want to load them from
somewhere else (let's say, using urllib, accessing GFS, going to Mars
and coming back, what if ...) That code does not scale if different
requirements are needed.

The one I mentioned does since it's based on XUnit philosophy and the
concept of loaders and GoF decorator pattern, and synergies between
classes thus promoting code reuse. So I wonder :

Why not to use dutest as the starting point for this dev ? What's
wrong with it ? It solves exactly the same problem ;)

.. [1] MultiTestLoader class
         (https://apps.sourceforge.net/trac/flioops/browser/py/trunk/utils/dutest.py?rev=83&marks=400-429#L399)

.. [2] PackageTestLoader class
         (https://apps.sourceforge.net/trac/flioops/browser/py/trunk/utils/dutest.py?rev=83&marks=434-532#L433)

.. [3] oop.test package (exdbc_* modules)
        (https://apps.sourceforge.net/trac/flioops/browser/py/trunk/utils/)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Se retira el BDFL ... ¿El fin de Python 3k?