[TIP] Guidelines for where to put tests & how to package them

[Michael Foord]

On 27/02/2010 19:15, C. Titus Brown wrote:
> On Fri, Feb 26, 2010 at 07:36:59PM +0000, Michael Foord wrote:
>    
>>> test command performs test discovery (as a last recourse AFAICR, /me
>>> cc-ing to P.J. Eby because probably he has further updates ;o) using a
>>> loader: setuptools.comand.test.xxx
>>>
>>>        
>> I'd rather follow up with Tarek (cc'd) as the lead maintainer of both
>> distribute (the maintained fork of setuptools) and of distutils(2).
>>      
> Aieeee, I'm trying to come up with a sensible convention BEFORE we bring
> it to the stdlib people ;).
>    

We have different goals in this discussion. You want a convention that 
can be used with existing versions of Python for pony-build, I am 
concerned with enshrining a good convention in the standard library. For 
what it is worth I think Tarek and I have arrived at a good solution for 
this (see below) with the painful caveat that I have to implement it for 
distutils2.

I *still* think that pony-build would be better served by adopting the 
standard library convention, and depending on unittest2 / distutils2 on 
the client to do that. :-) I probably won't mention this again...
>    
>>> so it would be very nice when defining packaging
>>> conventions, layout ... not to enforce the use of that protocol, so
>>> that if another discovery is used then it could still be compatible
>>> with the more abstract good practices. (all this said from my selfish
>>> position, I confess)
>>>        
>> Yes, but get_suite/test_suite is not supported, so distutils2 still
>> *should* use a unittest supported protocol. Probably Tarek and I need to
>> discuss how the setup.py test command should work for distutils2. It
>> should support non-unittest tests as well (I guess?) so specifying a
>> unittest suite is not necessarily ideal.
>>      
> YES, please let's support non-unittest tests.
>    

distutils2 will have tests and testrunner arguments in the setup.py for 
specifying any testrunner (non-unittest or otherwise) and a test 
package. If you specify tests without a test runner then unittest will 
be used, if you specify neither then discovery will be attempted.

With the load_tests protocol your test module or package (specified by 
tests) can return a unittest compatible suite even if tests are actually 
run under-the-hood with another framework. unittest will then only be 
used for reporting results.

distutils2 won't have a runtime dependency on unittest2 - except if it 
isn't available then test discovery won't be attempted (you will have to 
supply explicit tests/testrunner arguments).

>    
>> READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
>>      
> Really?  And that's at the BOTTOM of your e-mail? :)
>
>    
Awesome isn't it. It is by Cory Doctorow:

http://boingboing.net/2007/01/25/reasonableagreemento.html

Michael


> --titus
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.