[TIP] unitteset outcomes (pass/fail/xfail/error), humans and automation

[Robert Collins]

On Wed, 2009-12-16 at 15:15 +0000, Michael Foord wrote:
> On 15/12/2009 22:47, Robert Collins wrote: 
> > On Tue, 2009-12-15 at 16:24 -0500, Olemis Lang wrote:
> >   
> > > But anyway , it is often possible to wrap the execution of those TFs
> > > using customized instances of TestCase thereby recording the results
> > > provided by those frameworks using TestResult & Co.
> > > 
> > > PS: Does anybody knows about an exception to that «rule» ? 
> > >     
> > Well yes, at the moment you cannot preserve the full resolution because
> > the unittest module isn't very extensible.
> > 
> > See for instance what nosetests's ErrorClassPlugin does / recommends:
> > monkey patching the result object :- something totally incompatible if
> > the result object is e.g. a gui result, or a remoting result.
> >   
> 
> I'm definitely in favour of making unittest more modular and
> extensible, but am worried about creating a situation where there are
> multiple incompatible extensions. It seems like this could be a
> problem with the proposed outcomes.

Well, I haven't proposed an implementation yet ;). Consider though
that /right now/ we have multiple incompatible extensions: nose monkey
patches result objects, testtools permits registration, bzr until
recently reimplements TestCase.run (but now registers via testtools
API).

> I'm interested in this discussion but have never had the need for
> custom outcomes, so don't feel I can add very much here.

I think a very easy way to think about the problem is to imagine that
skip & expected fail were not in the standard library, and you wanted to
add them *as an extension*.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20091225/96ddf7bf/attachment.pgp>