[TIP] Guidelines for where to put tests & how to package them

[C. Titus Brown]

On Thu, Feb 25, 2010 at 06:03:24PM +1100, Robert Collins wrote:
> On Wed, 2010-02-24 at 21:01 -0800, C. Titus Brown wrote:
> > Hi all,
> 
> ..
> > Full source at http://github.com/ctb/SomePackage or downloadable
> > at
> > 
> >    http://lyorn.idyll.org/~t/transfer/SomePackage.tar.gz
> > 
> > Comments?  Thoughts?  Complaints?  Issues I missed?
> 
> The boilerplate 'run' isn't needed.
> 
> Rather than 'get_suite' define 'test_suite', then
> $ trial somepackage.tests will 'just work'
> As will
> #actually, there is a bug open on this one:
> $ python -m unittest somepackage.tests.test_suite
> # but this works:
> $ python -m subunit.run somepackage.tests.test_suite
> $ python -m testtools.run somepackage.tests.test_suite

I can't get that to run when I define a 'test_suite' attribute on
somepackage.tests -- what am I doing wrong?  Should test_suite be
a function or an iterable?

> If instead you define the load_tests hook, python 2.7 and 3.1 will work
> with
> $ python -m unittest somepackage   # discovery ftw
> 
> In older pythons the discover module can do this for you as well, I
> don't know its command line interface tough.

Yes, thanks for pointing this out!  Let me back up...

At the PyCon language summit, distutils2 was born as a clean version of
Distribute that will *replace* distutils in the future.  It will be part of
Python 3.3 going forward, and it will be an installable PyPI package from 2.4
on up.

My goal is to define a standard policy for running developer tests, and perhaps
post-install tests, in the Hitchhiker's Guide to Packaging
(http://guide.python-distribute.org/).  I'd like to outline a mechanism that
works back to 2.4 with regular distutils, because there are still many people
running and producing 2.4 and 2.5-compatible software; this fits with Tarek
Ziade's goal to produce a 2.4-compatible distutils2.  Whatever policy we choose
should work cleanly with regular distutils as well as Distribute/distutils2.

This means we can't rely on anything other than unittest and module-execution
mechanisms present back in Python 2.4.

I also want the mechanism to be compatible with installed packages, so
that my users can run my code and send me diagnostics.

And of course it needs to work with all (or at least most) of the major 3rd
party unit test frameworks.

Since

  % python -m <something>

works fine on 2.4+ for both dev and install environments, it seems like
a good standard.  The question of what <something> is does need to be
answered :).

And, when a setup.py is present (i.e. dev environment),

  % python setup.py test

is going to be the new standard in the brave new distutils2 world.

Does that makes more sense as a backstore? ;)

thanks,
--titus
-- 
C. Titus Brown, ctb at msu.edu