[TIP] Result protocol
jnoller at gmail.com
Sat Apr 11 11:39:01 PDT 2009
On Sat, Apr 11, 2009 at 2:35 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
>> Building on this, and Scott's point -
>> id: STR
>> result: STR: PASS | FAIL | ERROR | SKIP | UNKNOWN | KILLED (or
>> result_id: STR: 0 | 1 | 255 | ... others
>> Ideally, the ID is a simple int, ala Unix error codes. In the past,
>> I've used something like ERROR: 300, SKIP: 301, or other. I try not to
>> use ones low enough to be relevant to real exit codes as it could be
>> useful to actually pass back the exit code of the test command itself
>> - on the fence about that though.
> Having result *and* result_id seems like a recipe for confusion (what if
> they contradict). Keeping it as a string keeps it human readable.
Fair enough, I can jam the ID (Yay numbers!) into the additional
section if I really want it.
> But with most testing tools the traceback *is* the diagnostic information.
> It isn't *obvious* where it should go in the schema you have below.
Could we call it something other than traceback? Otherwise, I'm OK
with putting it in the test: section, I was on the fence about it.
>> Hmm, stdout and stderr are a PITA - I dealt with a test in the past
>> that created log files several mb in size, logjamming it in this would
>> suck. How about something like "last N lines" for both? Or we make it
>> optional, ala something like:
>> build_id: STR
>> machine_id: STR (uuid?)
>> id: STR
>> result: STR
>> result_id: STR
>> start: FLOAT (time.time())
>> stop: FLOAT (time.time())
>> total_time: FLOAT (seconds)
>> coverage_info: Big str
>> stdout: Big Str
>> stderr: Big str
> OK - although I would still rather see one obvious place for the traceback
> (whatever it may be called) for failure information.
See above, I'm iffy on calling it traceback though.
More information about the testing-in-python