[TIP] Result protocol (was: Nosejobs!)
jnoller at gmail.com
Sat Apr 11 20:39:12 PDT 2009
On Sat, Apr 11, 2009 at 11:03 PM, Robert Collins
<robertc at robertcollins.net> wrote:
> About right yes. There are things missing [that is uses the xUnit
> interface deliberately, as a key design goal], but at the heart of it
> you are very close. There is one thing in particular that is important -
> subunit serialises *activity* not results.
Yup, sorry I left that out - but I think we both know by now I don't
want to be stuck playing in the xUnit sandbox.
> If you need timeout then you are approaching the problem in a different
> way - and that will make it harder IMO. Note that subunit does not need
> timeout because of the bracketing approach taken to describing activity.
> An interrupted test is detected automatically (if its is killed), and
> you can enforce a timeout policy both wherever the test is executing,
> and where you are reading test activity from.
Like I said, there are parts to the approach which are attractive, but
yes, I am approaching it a different way. We have two different
centers of design.
> Actually no - you have been talking about wire protocols, not about
> objects. And, the wire protocols that have been discussed - except for
> TAP and subunit - do not appear to fit _at all_ with the unittest object
> model. And TAP itself fits rather badly [see tap2subunit for the
> gymnastics needed].
Yes. I saw that bit of joy (tap2subunit) - but I mentioned object
earlier (which is how I tend to think) and michael made me stop it :)
>> I can't disagree here, which is why I am proposing using YAML (let's
>> ignore JSON for the moment) and/or XML. It's easily extendable, does
>> not require custom parsers, and convey a near unlimited amount of
>> information. Additionally, they go on the wire pretty nicely.
> But proposing yaml/json/xml *is* talking about wire protocols. I repeat:
> The *interesting* problem is the object protocol, not the wire
I think this is a sign of the differing approaches, I don't *care*
about the *internal* objects within the framework/executor. I want a
common results format - that goes on the wire simply enough. The
objects happen to be a set of YAML/JSON with some required fields.
> Until unittest can represent a standard way for
> accepting/exposing/accessing/whatever all the activity and data that a
> test running remotely may generate, designing a wire protocol is
You fight the changing of unittest fight, I'll stick to nose for now :)
> So, I've said before that I am open to changing the syntax. Writing a
> parser is trivial though - just having a dict for each action from the
> wire [which is the most any of the mentioned encodings do] doesn't
> provide glue to a given languages unittest equivalent. *No matter what
> approach is taken* language specific glue needs to be written. I'm not
> trying to handwave away the issue of the current approach needing a
> parser written - but honestly, the state machine for the parser is a
> small fraction of the total code base.
It's code that doesn't have to be written though (the parser). I don't
know that *I* want you to change your syntax, it works for you. I just
have a preference on not using any custom syntax or sticking to xUnit
> So subunit can be extended to convey more data - I just don't want to do
> that before being able to do it all in-python. Otherwise there _will_ be
> a mismatch, and I can assert from experience that we'll get it wrong and
> end up having just-another-not-reusable-protocol. Something that subunit
> avoids at the moment.
You'd have to take that up with someone working on Unit test (looks at
michael) for core python, I'm not going near that bikeshed ;) I'll
stick to my multiprocessing module.
> I was younger then ;). OTOH, pynie (parrot's python compiler) have
> recently adopted subunit to run their bootstrap tests, and they just
> copied the __init__ into their source tree, which is very convenient for
Yeah, but its in the __init__ ucky!111 (Of course, multiprocessing
does a bunch of stuff in __init__ I haven't had a chance to remove
yet, so *shrug*)
More information about the testing-in-python