[TIP] decorators in testing : warning support for TestCase

Douglas Philips dgou at mac.com
Mon Apr 6 08:17:08 PDT 2009

On or about 2009 Apr 6, at 9:21 AM, Olemis Lang indited:
> - The recent thread about higlighting the tests that were previously
>  run but didnt show up at testing time ... this is not precisely an
>  error, but perhaps means that something weird was going on, or
>  maybe not ;).

Perhaps, but I don't think that should be part of the regression  
runner itself.

> - The same use case we are talking about ... if setUp fails there are
>  mainly three options to consider :
>  * signal a failed test case ... but IMHO this *SHOULD NOT* be used
>    since the test itself didnot fail ... it just couldnt be
>    performed ... so a better approach is ...

Agreed that is not an option.

>  * ... to signal a test failure, indicating that the test code was
>    not accurate and needs to be fixed. But maybe this is not the
>    case. Perhaps under certain circumstances the lack of resources
>    is not a test failure at all, but a consequence of a specific
>    config ... and in this case maybe it is better ...

Yes, and we use "Skip" in this case, though in our version of the  
unittest framework we use an eligibility test before even getting to  
setUp, and that catches most of these "skip in setUp" kinds of cases.

>  * ... to log a warning message ... ;)

One other thing we do in setUp is protect our code with asserts to  
indicate that we need to go back and write more test code. For  
example, if the product specs says: "Device can do X", we need to test  
X. However, if the current hardware can't do X, we assert that to  
protect our test from producing an false-postive/PASS. Sure, we could  
speculatively write code to test X, but if we can't run that code, it  
is highly suspect and likely to require bug fixes when X is actually  
made available. Instead of wasting time on code we can't run, we  
protect our tests with asserts. Our philosophy over all is that a test  
FAILure is a bug in the device-under-test. A test ERRor is a bug in  
the test itself, not the device.


More information about the testing-in-python mailing list