[TIP] Guidelines for where to put tests & how to package them

[Brett Cannon]

On Tue, Mar 2, 2010 at 15:14, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> On 02/03/2010 19:21, holger krekel wrote:
>
>> On Fri, Feb 26, 2010 at 21:53 +0000, Michael Foord wrote:
>>
>>
>>> On 25/02/2010 05:01, C. Titus Brown wrote:
>>>
>>>
>>>> Hi all,
>>>>
>>>> here at PyCon there have been a lot of packaging discussions, so I
>>>> thought
>>>> I'd spend a bit of time outlining some suggestions for where to put
>>>> tests and how to run them.  It's been a bit of a thorn in the side of
>>>> (among other things) continuous integration systems that there's no
>>>> standard way to run Python tests... so let's fix that!
>>>>
>>>> I've produced a simple draft proposal&   example where you put your unit
>>>> tests
>>>> under a package dir, somepackage/tests/.
>>>>
>>>> You can run these tests with
>>>>
>>>>    % python -m somepackage.tests.run
>>>>
>>>> and you can also do (if setuptools/distribute is installed)
>>>>
>>>>    % python setup.py test
>>>>
>>>>
>>>>
>>> If you're happy to depend on distutils2 and unittest2 then very soon
>>> there will be a standard "setup.py test" command that will continue to
>>> work with future versions of Python. It will probably default to
>>> unittest test discovery but allow you to provide a specific 'tests'
>>> argument (either a module or suite) and a 'testrunner' (test_runner ?)
>>> argument to specify a non-unittest runner if needed (testrunner defaults
>>> to unittest.TextTestRunner, well actually unittest2.TextTestRunner).
>>>
>>>
>> (i am on low bandwidth and vacation, just a quick note)
>> I'd appreciate it if we had a more general mechanism
>> than a unittest TestCase and a test runner.  Maybe just a callable
>> with some directory argument that is to discover and run the tests?
>> After all there are non-python tests and non-TestCase tests these
>> days and introducing a standard should be general enough for that.
>>
>>
>
> There are three protocols suggested (and it would be nice to get that down
> to two... :-), none of which require tests to be TestCase based - although
> one of the protocols requires a test suite like object runnable with a
> TextTestRunner. As I explain below this is a very minimal requirement. The
> three protocols we have discussed are:
>
> Packages provide a test sub-package with a run module that is able to
> collect and run the tests when executed with:
>
>    python -m package.test.run
>

Can I suggest this change to package.test.__main__? The reason for this is
that starting in Python 2.7 and already in Python 3.1 is support for
executing packages through runpy. The __main__ module in a package is what
gets called and executed as '__main__'. It works out really nicely as the
execution line would become::

  python -m package.test

Which does not require remembering the specific name of the module to
execute. Plus it is backwards-compatible with older versions of Python that
support -m::

  python -m package.test.__main__

I am already using this in the stdlib for importlib and I am quite happy;
it's become my preferred way to execute tests by hand when doing
development. Plus it provides a nice place to provide command-line parsing
when I have to tweak something (in my case using __import__ instead of
importlib for quick compatibility tests through a --builtin argument).

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20100302/47e284ab/attachment.htm>