[TIP] Result protocol (was: Nosejobs!)

[Jesse Noller]

On Sat, Apr 11, 2009 at 11:03 PM, Robert Collins
<robertc at robertcollins.net> wrote:

>
> About right yes. There are things missing [that is uses the xUnit
> interface deliberately, as a key design goal], but at the heart of it
> you are very close. There is one thing in particular that is important -
> subunit serialises *activity* not results.

Yup, sorry I left that out - but I think we both know by now I don't
want to be stuck playing in the xUnit sandbox.


> If you need timeout then you are approaching the problem in a different
> way - and that will make it harder IMO. Note that subunit does not need
> timeout because of the bracketing approach taken to describing activity.
> An interrupted test is detected automatically (if its is killed), and
> you can enforce a timeout policy both wherever the test is executing,
> and where you are reading test activity from.

Like I said, there are parts to the approach which are attractive, but
yes, I am approaching it a different way. We have two different
centers of design.

> Actually no - you have been talking about wire protocols, not about
> objects. And, the wire protocols that have been discussed - except for
> TAP and subunit - do not appear to fit _at all_ with the unittest object
> model. And TAP itself fits rather badly [see tap2subunit for the
> gymnastics needed].

Yes. I saw that bit of joy (tap2subunit) - but I mentioned object
earlier (which is how I tend to think) and michael made me stop it :)

>> I can't disagree here, which is why I am proposing using YAML (let's
>> ignore JSON for the moment) and/or XML. It's easily extendable, does
>> not require custom parsers, and convey a near unlimited amount of
>> information. Additionally, they go on the wire pretty nicely.
>
> But proposing yaml/json/xml *is* talking about wire protocols. I repeat:
>  The *interesting* problem is the object protocol, not the wire
> protocol.

I think this is a sign of the differing approaches, I don't *care*
about the *internal* objects within the framework/executor. I want a
common results format - that goes on the wire simply enough. The
objects happen to be a set of YAML/JSON with some required fields.

> *****
> Until unittest can represent a standard way for
> accepting/exposing/accessing/whatever all the activity and data that a
> test running remotely may generate, designing a wire protocol is
> premature.
> *****

You fight the changing of unittest fight, I'll stick to nose for now :)

> So, I've said before that I am open to changing the syntax. Writing a
> parser is trivial though - just having a dict for each action from the
> wire [which is the most any of the mentioned encodings do] doesn't
> provide glue to a given languages unittest equivalent. *No matter what
> approach is taken* language specific glue needs to be written. I'm not
> trying to handwave away the issue of the current approach needing a
> parser written - but honestly, the state machine for the parser is a
> small fraction of the total code base.

It's code that doesn't have to be written though (the parser). I don't
know that *I* want you to change your syntax, it works for you. I just
have a preference on not using any custom syntax or sticking to xUnit
rules.

> So subunit can be extended to convey more data - I just don't want to do
> that before being able to do it all in-python. Otherwise there _will_ be
> a mismatch, and I can assert from experience that we'll get it wrong and
> end up having just-another-not-reusable-protocol. Something that subunit
> avoids at the moment.

You'd have to take that up with someone working on Unit test (looks at
michael) for core python, I'm not going near that bikeshed ;) I'll
stick to my multiprocessing module.

> I was younger then ;). OTOH, pynie (parrot's python compiler) have
> recently adopted subunit to run their bootstrap tests, and they just
> copied the __init__ into their source tree, which is very convenient for
> users.

Yeah, but its in the __init__ ucky!111 (Of course, multiprocessing
does a bunch of stuff in __init__ I haven't had a chance to remove
yet, so *shrug*)

jesse