[TIP] Fail-fast testing?

[Michael Foord]

On 05/03/2010 21:43, Robert Collins wrote:
> On Fri, 2010-03-05 at 11:40 -0500, J. Cliff Dyer wrote:
>    
>> Would it be possible to get something like the fail-fast features that
>> have been incorporated into django 1.2 included in unittest somehow?
>>
>> There are two improvements that I think could be generally useful.
>>
>> http://code.djangoproject.com/changeset/12034 provides a Ctrl-C override
>> (using the signals module), which prevents the test runner from dying
>> with an exception on Ctrl-C, and instead finishes the current test, and
>> returns the results up to that point.  The output looks like this:
>>
>> $ ./manage.py test
>> [Snipped test setup]
>> .........................^C<Test run halted by Ctrl-C>  .
>> ----------------------------------------------------------------------
>> Ran 26 tests in 1.488s
>>
>> OK
>> Destroying test database 'default'...
>>
>>
>> http://code.djangoproject.com/changeset/11843 provides a --failfast
>> option which exits the test run after the first failure, showing all
>> test output up to that point.  Both of these are great options to have
>> available when working with large test suites.
>>
>> I'd be happy to put in some legwork if there were interest in having
>> this functionality available in unittest.
>>      
> I think the ctrl-c handling could be nice; need to be careful though -
> some programs test their own ctrl-c handling; don't stomp on them.
>    

The fast fail is easy.

Adding a signal handler that stores the ctrl-c and checks for it between 
tests is easy enough. Is signal.signal(signal.SIGINT, handler) 
sufficiently cross-platform? (With a second ctrl-c in a single test 
calling the default handler.)

How do you recommend doing it in such a way that it doesn't interfere 
with tests doing their own ctrl-c handling?

Such tests should really install their own handler and then restore the 
original - so it *shouldn't* be a problem, unless the tests don't 
install their own handler and send a SIGINT some other way - ideas on 
how to handle this?

It would be really nice to have the ctrl-c handling default to on, but 
perhaps an easy way of handling this is just have it default to off.

Michael

> Both of these things should be very easy though, as shouldStop is
> already part of the result contract.
>
> -Rob
>    
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>    


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20100305/1873e57f/attachment.htm>