[TIP] unittest2, distribute, and doctests

Olemis Lang olemis at gmail.com
Mon Jul 5 13:12:24 PDT 2010

On 6/7/10, Barry Warsaw <barry at python.org> wrote:
> Hey guys, I have a few quick questions about unittest2.  Apologies if these
> have been discussed before.

More or less i.e. not exactly this subject , but others related to
this one . What follows may be interesting but may also be OT , its

> Distribute adds the ability to run 'python setup.py test' by including the
> `test_suite` keyword to your setup().

... like the old setuptools-way (maybe better ;o)

> What's the best way to integrate
> unittest2 and distribute to enable the former to do test discovery when run
> via the setup.py?

pass , I don't use it ...

> `python setup.py test` supports a -m and -s option to filter (minimally) the
> tests that are run.  Can unittest2's richer command line options be
> supported?
> (This may be more of a distutils/setuptools/distribute question.)

This is a very good question !!!

Considering the way test discovery has been implemented I really think
that's very hard to do this . The reasons why this happens can be
found in long threads (me and Michael mostly ;o) about why I'd not use
the approach that's been employed in there .

OTOH , dutest is prepared to perform this kind of discovery 'cause it
provides PackageTestLoader class and decouples test discovery
completely from other testing aspects . This class supports multiple
matching patterns. Right now :

  - supports file name patterns (e.g. fnmatch)
  - supports regular expressions (i.e. re)
  - it's possible to use different loaders to load tests
     from single modules (e.g. doctests,TestCase(s) ,
     customized doctests using exotic output checkers
     and runners ... )
  - more ... don't have much time right now ...

In the future I plan to add support for FSL matching (File Selection
Language, originally written for automatic backup software AFAICR
according to the author ;o) once I perform some refactorings in there
. Of course , the impact of implementing this over existing
implementation is minimal . After doing this

  - Much finer granularity will be possible
  - Custom predicates will be available
  - The (my) idea is to build a pluggable
    architecture for adding predicates so that ,
    for instance, users will be able to specify
    (describe) complex discovery scenarios like
    this one

"Load tests in files under test directory but skip all those inside
test/db, test/security/msad, but only if previous per-test execution
times are over 2s or if the target test module contains more than
200,000 test cases . For each such module found, extract unittest
TestCases, and doctests, and ... but I want the later to be reported
using UDIFF flag and executed with NORMALIZE_WHITESPACE always set"

More complex situations are possible, but so far it's enough to
capture the idea ;o) The only thing is that IMHO the current
implementation details are an obstacle in order to get this done (as I
mentioned before in old thread ...)

In order to get this done right now the only thing that's needed and
missing (besides the FSL thing ;o) is to connect dutest (i.e. write
the distutils command) providing access to this class (I've not done
this already 'cause I've been extremely busy, but my motivation of
developing in this field is increasing ;o) .

I suppose that subclassing the original `test` command might be an option ;o)

> unit2 -p takes shell globs, but I really like zope.testrunner's regexp-based
> filters.  Any chance unittest2 will support something like that?  One of the
> reasons I like zope's test discovery is because it works much better with
> doctests.  E.g. I can do (after a buildout):
>     % bin/test -vv -t using
> and it will only run my using.txt doctest.  Note that I have a test_docs.py
> module that does individual .txt doctest discovery, wrapping each one in a
> DocFileSuite with appropriate optionflags, and setUp.


I had never seen this one , what's this ?

> unittest2
> eventually grew better support for doctests.

/me saying since long time ago , with respect to dutest ...

All this, like I said is JFTR, JFYI and pseudo-OT



Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:

More information about the testing-in-python mailing list