[TIP] Guidelines for where to put tests & how to package them

[Michael Foord]

On 25/02/2010 05:01, C. Titus Brown wrote:
> Hi all,
>
> here at PyCon there have been a lot of packaging discussions, so I thought
> I'd spend a bit of time outlining some suggestions for where to put
> tests and how to run them.  It's been a bit of a thorn in the side of
> (among other things) continuous integration systems that there's no
> standard way to run Python tests... so let's fix that!
>
> I've produced a simple draft proposal&  example where you put your unit tests
> under a package dir, somepackage/tests/.
>
>    

I've been using 'somepackage/test/', but 'somepackage/tests/' is fine 
with me and it would be better to have one-way-to-do-it.

Note that 'test/' should also be a package (should contain an 
__init__.py). I also agree that it is good practise not to have any 
dependencies on your test package(s). A top level 'test/' package for 
functional / integration tests also sounds fine.

In Python 2.7 / 3.2 you will be able to autodiscover and run unittest 
based tests with:

python -m unittest discover

See: http://docs.python.org/dev/library/unittest.html#test-discovery

For Python 2.4-2.6 you can get this functionality (and all the other 
nice new functionality in unittest) with the unittest2 package. The way 
you use the command line is with the 'unit2' script instead of 'python 
-m ...'. See:

http://pypi.python.org/pypi/unittest2

If you *just* want test discovery for unittest, without having to switch 
to unittest2, you can use the discover module. Tests are then discovered 
with "python -m discover":

http://pypi.python.org/pypi/discover

So lots of options for autodiscovery and running of unittest based tests.

I actually quite like the idea of having a run module in your tests so 
that "python -m somepackage.test.run" always works. This allows a single 
defined entry point *whichever* test framework you are using. The other 
mechanisms discussed depend on the user *knowing* which framework was 
used in order to be able to invoke the tests (although if you have 
external dependencies they will still need to be installed of course).

All the best,

Michael

> You can run these tests with
>
>    % python -m somepackage.tests.run
>
> and you can also do (if setuptools/distribute is installed)
>
>    % python setup.py test
>
> ---
>
> Full source at http://github.com/ctb/SomePackage or downloadable
> at
>
>     http://lyorn.idyll.org/~t/transfer/SomePackage.tar.gz
>
> Comments?  Thoughts?  Complaints?  Issues I missed?
>
> thanks,
> --titus
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.