[TIP] Guidelines for where to put tests & how to package them

[holger krekel]

On Fri, Feb 26, 2010 at 21:53 +0000, Michael Foord wrote:
> On 25/02/2010 05:01, C. Titus Brown wrote:
>> Hi all,
>>
>> here at PyCon there have been a lot of packaging discussions, so I thought
>> I'd spend a bit of time outlining some suggestions for where to put
>> tests and how to run them.  It's been a bit of a thorn in the side of
>> (among other things) continuous integration systems that there's no
>> standard way to run Python tests... so let's fix that!
>>
>> I've produced a simple draft proposal&  example where you put your unit tests
>> under a package dir, somepackage/tests/.
>>
>> You can run these tests with
>>
>>    % python -m somepackage.tests.run
>>
>> and you can also do (if setuptools/distribute is installed)
>>
>>    % python setup.py test
>>    
>
> If you're happy to depend on distutils2 and unittest2 then very soon  
> there will be a standard "setup.py test" command that will continue to  
> work with future versions of Python. It will probably default to  
> unittest test discovery but allow you to provide a specific 'tests'  
> argument (either a module or suite) and a 'testrunner' (test_runner ?)  
> argument to specify a non-unittest runner if needed (testrunner defaults  
> to unittest.TextTestRunner, well actually unittest2.TextTestRunner).

(i am on low bandwidth and vacation, just a quick note)
I'd appreciate it if we had a more general mechanism
than a unittest TestCase and a test runner.  Maybe just a callable 
with some directory argument that is to discover and run the tests? 
After all there are non-python tests and non-TestCase tests these 
days and introducing a standard should be general enough for that. 
cheers & thanks,
holger


> This means that no *specific* layout for tests need to be mandated, so  
> long as it is compatible with test discovery or you do the test  
> collection yourself with the 'tests' argument.
>
> With this in place (which it isn't yet) it is unlikely that unittest /  
> distutils will adopt an official 'recommended' test layout, but don't  
> let that stop you... ;-)
>
> FWIW I would rather see ponybuild use distutils2 / unittest2 (i.e. the  
> recommended [1] community tools), but I understand your reluctance to  
> take a dependency on anything other than a single script on the client  
> side. A bootstrap script for installing dependencies would be one way  
> round this.
>
> All the best,
>
> Michael Foord
>
> [1] Recommended by me...
>
>> ---
>>
>> Full source at http://github.com/ctb/SomePackage or downloadable
>> at
>>
>>     http://lyorn.idyll.org/~t/transfer/SomePackage.tar.gz
>>
>> Comments?  Thoughts?  Complaints?  Issues I missed?
>>
>> thanks,
>> --titus
>>    
>
>
> -- 
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
> READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>

--