[TIP] Result protocol
jnoller at gmail.com
Sat Apr 11 14:27:34 PDT 2009
On Sat, Apr 11, 2009 at 3:46 PM, Scott David Daniels
<Scott.Daniels at acm.org> wrote:
> Jesse Noller wrote:
>> id: STR
>> result: STR: PASS | FAIL | ERROR | SKIP | UNKNOWN | KILLED (or
>> result_id: STR: 0 | 1 | 255 | ... others
>> Ideally, the ID is a simple int, ala Unix error codes. In the past,
>> I've used something like ERROR: 300, SKIP: 301, or other. I try not to
>> use ones low enough to be relevant to real exit codes as it could be
>> useful to actually pass back the exit code of the test command itself
>> - on the fence about that though.
> I don't know what UNKNOWN, KILLED, and TIMEOUT mean (in some "precise" way).
> like to see that defined for a protocol.
UNKNOWN: Unknown/un-identifiable exit, either it checked out (and
wasn't killed) or the status could not be gotten. Could also be None
KILLED: The executor killed the child test, this could be due to
taking too long, sudden preemption, or some other thing.
TIMEOUT: Test timed out - you could possibly use this instead of
KILLED to define a test which has exceeded a built in timer (say, 3
hours) and had to be killed.
>> build_id: STR
>> machine_id: STR (uuid?)
>> id: STR
>> result: STR
>> result_id: STR
>> start: FLOAT (time.time())
>> stop: FLOAT (time.time())
>> total_time: FLOAT (seconds)
> The presence of start, stop, and total_time seems to ask for
> normalization. I'd drop one (since it can be calculated from the others).
> Perhaps drop stop, since total may have more precision.
Personally, I'd drop total, stop and start would be fine w/me.
More information about the testing-in-python