[TIP] Result protocol

[Jesse Noller]

On Sat, Apr 11, 2009 at 11:36 PM, Douglas Philips <dgou at mac.com> wrote:

> Well, Pandokia showed some concrete examples I found pretty compelling
> for simplicity.
> Do you have something in json we can do a bake-off, so to speak, with?

Sure, when I get a chance, I'll build up some mock YAML stuff. Most
YAML is correct JSON, so I'm not concerned about the JSON case right
now.

>> Flat text/custom parsing sucks. That's my steak on the ground.
>
> And bacon. Don't forget the bacon! :)
>

mmmm. Bacon. Mayhaps we call it the Bacon Testing Protocol? Delicious.

> The Pandokia format is very close to .ini style. Hypothetical query:
> if it were .ini, would that make any difference (it wouldn't be custom
> any more, n'est-ces pas?)
>

And INI files have some annoying as hell restrictions, YAML support
conversion from items within the file to native data types, such as
lists, ints, and so on. I don't want to have to parse it and do the
conversion myself. Many people have written custom parsers to do this
conversion for you, such as Michael (configobj) but again, why not use
something which support conversion to native data types so that the
job is easier on the parser and emitter?

> If we all want to interoperate, and the protocol is covered by GPL?

Personally, anything I do will be under a BSDish license. Business reasons.

> We are indeed talking about a custom DSL, whether it is written with
> YAML/JSON/INI/<ad nauseum> since the syntax doesn't say what anything
> means. A bit of a red-herring here.

No, we're talking about a file with some custom fields. That's worlds
different than a DSL. And true, the syntax doesn't "say what it means"
- but it does provide the type of the object, which goes a long way.

> Maybe we should work out what this DSL "talks about" independent of
> the serialization, and consider the serialization/formatting
> separately? Or should we hash out first if it is going to be Latin-1
> or UTF-8 or ??

That's what my goal is/was: define the fields in the file. It just so
happens my example is somewhat valid YAML.

job_id: UUID
recv_time: FLOAT
build_id: STR
machine_id: STR (uuid?)
execution_line: STR

run_stats:
   total_tests: INT
   total_pass: INT
   total_fail: INT
   total_other: INT
   start_time: FLOAT
   stop_time: FLOAT
   total_time: FLOAT

test_cases:
   test:
       id: STR
       result: STR
       result_id: STR
       start: FLOAT (time.time())
       stop: FLOAT (time.time())
       total_time: FLOAT (seconds)
       additional:
           coverage_info: Big str
           stdout: Big Str
           stderr: Big str

extended:
   requestor_id: INT
   artifacts_uri: <your path here>
   artifacts_chksum: MD5
   configuration_file: (the config file passed in in my case)

Roughly parsed into { job_id : value, test_cases : { ... } } and so
on. See http://pyyaml.org/wiki/PyYAML. The values on the other side of
:  characters are converted to native data types and things like
lists, etc.

jesse