[TIP] Result protocol / pass-fail-error

[Olemis Lang]

On Mon, Apr 13, 2009 at 11:34 AM, Mark Sienkiewicz <sienkiew at stsci.edu> wrote:
> Jesse Noller wrote:
>>>> 2> There is the generic concept of a "test result" - in my mind, this
>>>> is the simple "PASS/FAIL/ERROR" concept of a test which is completed.
>>>>
>>>>
>>> Agreed - there should be a single result element with values PASS / FAIL /
>>> ERROR.
>>>
>>
>> Building on this, and Scott's point -
>>
>> test_cases:
>>     test:
>>         id: STR
>>         result: STR: PASS | FAIL | ERROR | SKIP | UNKNOWN | KILLED (or TIMEOUT)
>>         result_id: STR: 0 | 1 | 255 | ... others
>>
>
> You do not need to define all the possible values for result.  Instead,
> you can define the most common, but allow other user-defined values.
>

+1 ... my suggestion according to my own experience is to have a
one-level hierarchy like the following :

- ERROR
- FAIL
  * KILLED
  * TIMEOUT
  * UNKNOWN
- PASS
  * SKIP

thereby allowing to refine ERROR, FAIL & PASS

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article: