[TIP] Guidelines for where to put tests & how to package them

[Michael Foord]

On 27/02/2010 19:32, C. Titus Brown wrote:
> On Sat, Feb 27, 2010 at 07:27:34PM +0000, Michael Foord wrote:
>    
>> On 27/02/2010 19:15, C. Titus Brown wrote:
>>      
>>> On Fri, Feb 26, 2010 at 07:36:59PM +0000, Michael Foord wrote:
>>>
>>>        
>>>>> test command performs test discovery (as a last recourse AFAICR, /me
>>>>> cc-ing to P.J. Eby because probably he has further updates ;o) using a
>>>>> loader: setuptools.comand.test.xxx
>>>>>
>>>>>
>>>>>            
>>>> I'd rather follow up with Tarek (cc'd) as the lead maintainer of both
>>>> distribute (the maintained fork of setuptools) and of distutils(2).
>>>>
>>>>          
>>> Aieeee, I'm trying to come up with a sensible convention BEFORE we bring
>>> it to the stdlib people ;).
>>>        
>> We have different goals in this discussion. You want a convention that
>> can be used with existing versions of Python for pony-build, I am
>> concerned with enshrining a good convention in the standard library. For
>> what it is worth I think Tarek and I have arrived at a good solution for
>> this (see below) with the painful caveat that I have to implement it for
>> distutils2.
>>
>> I *still* think that pony-build would be better served by adopting the
>> standard library convention, and depending on unittest2 / distutils2 on
>> the client to do that. :-) I probably won't mention this again...
>>      
> No, you mistake me: pony-build (and experience with other CI systems) makes me
> want to take it on, but I would like to have something sensible re testing in
> the Hitchhiker's Guide to Packaging -- which defines packaging practice from
> Python 2.4 onwards.
>
> So where our goals differ is that you want to define a new convention
> going forward (2.7+, 3.3+) and I want to document some convention that's
> "good enough"&  backwards compatible to 2.4.
>    

I will strongly argue, and suspect Tarek will agree, that the 
recommended practise for Python 2.4+ users is to use distutils2. That 
way they can use the conventions that will just-work (tm) in future 
versions of Python. distutils2 and unittest2 are both maintaining 
compatibility with Python 2.4 specifically so they can be used in this way.


> [snip...]
>> With the load_tests protocol your test module or package (specified by
>> tests) can return a unittest compatible suite even if tests are actually
>> run under-the-hood with another framework. unittest will then only be
>> used for reporting results.
>>      
> OK, this still seems a bit restrictive; as we've discussed, unittest is
> full of snakes that won't ever (or at least soon) be cleared out completely,
> for the sake of backwards compatibility.  So I'd like to avoid enshrining
> load_tests as a convention.
>    

load_tests is already a convention. This particular hook allows you to 
interoperate with unittest whilst avoiding most of the snakes. If you 
have *specific* problems with this approach then I'd like to hear about 
them. :-)

Anyway, regardless of whether you think it is a good thing to use it or 
not this hook already exists in the new and improved unittest and we 
certainly can't *stop* people using it this way if they want.

Michael

-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.