[TIP] unitteset outcomes (pass/fail/xfail/error), humans and automation
Robert Collins
robertc at robertcollins.net
Tue Dec 15 12:29:56 PST 2009
On Tue, 2009-12-15 at 09:45 -0500, Olemis Lang wrote:
> And also to treat then uniformly . I mean I am using testing framework
> X and suddenly switch to TF Y and that's transparent for all apps
> processing the test results (CI slaves or whatever ;o) ...
I don't think that such a change can be transparent in all cases. The
more transparent the better, but as a trivial example, does the
framework use test 'ids' or some human description, or the object path
to the test?
> > Well, on the standardising output front, have you looked at subunit ? :)
> > It aims at precisely that, with current included support for outputtin
> > it from pyunit, shunit, cppunit, 'check' (A C xUnit implementation),
> > converting to junit's xml format (which nearly all CI tools and many
> > IDE's understand) :)
> >
>
> ... the serialization of the test results is another subject , but could be
> simpler if all TFs report test results following a well-known
> (model | pattern | standard | specification ...)
CouldaWouldaShoulda :- testing frameworks don't all follow xUnit, and
even if they did, xUnit is a moving target: for all that people have
written books, xUnit is evolving and changing continually in the many
implementations of the basic idea.
> > I think the API interactions for the things you mention are well
> > addressable, though in a few different ways.
>
> Yes, but let's go step by step and focus on test outcomes first , isn't it ?
> ;o)
well, second yes, but front of the queue now.
-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20091216/dfee2d14/attachment.pgp>
More information about the testing-in-python
mailing list