[TIP] Result protocol / problems being solved (really)
Victoria G. Laidler
laidler at stsci.edu
Tue Apr 14 14:32:57 PDT 2009
Thanks Laura, this is very helpful.
Laura Creighton wrote:
> Some things I've read last week ....
> Some people have tests that are expected to run for a particular amount
> of time. They want a special way to tell when the test didin't run
> for that amount of time.
Implication: Test duration must be derivable from protocol.
> Some people want a way to indicate that they expect a test to fail
> in a certain way, so that when it fails in a different way they can
> make those differences stand out.
Implication: Finer-grained status than Pass/Fail.
> Some people are testing hardware. I think that Error (meaning
> broken test or execution env) won't be enough for them. They need to
> distinguish between 'my hardware didn't work properly' and 'something
> is wrong in this test' and 'this hardware didn't have the correct
> software loaded on it before we even tried to run this test'.
> Whether this last is a special case of the second, I am not sure.
Implication for possibly both of these: Custom statuses must be permitted.
More information about the testing-in-python