[TIP] test results tracking, etc...

[Douglas Philips]

On or about 2009 Apr 5, at 11:21 PM, C. Titus Brown indited:
> On Sun, Apr 05, 2009 at 09:22:32PM -0400, Douglas Philips wrote:
> -> Actually if you write up the docs and document the formats  
> needed, I
> -> like to see that spark a discussion here. I know Titus is in  
> favor of
> -> simplest-thing-that-could-possibly work, so I suspect that will be
> -> Pass/Fail. Personally I prefer to hard-coded valus even in simple
> -> systems. (The simplest code I know how to write is always "table-
> -> driven" and usually doesn't need to know or care about specific  
> values.)
>
> Sadly not everyone on this list agrees that my opinion matters above  
> all
> else, but I'm sure that pass/fail would be a good baseline ;).  As I
> recall Pandokia had some interesting stuff between pass and fail like
> "not loaded for resource reasons"...

Sigh, I meant to say "I prefer -not- to hard-code values even in  
simple..."
In our processing (so far), all we need is distinct values that go  
into buckets.
Currently we make a fuss about certain non-success status codes in the  
regression runner, but that is what we're trying to fix. Dragnet:  
"Just the facts" - the test runner should report on the status. Other  
tools downstream from there should be deciding what those results  
mean, perhaps in a historical context, perhaps not.
So I agree that simple is better. To me, table-driven is simpler than  
hard-coded values such as Pass/Fail. :)

-Doug