[TIP] decorators in testing : warning support for TestCase
dgou at mac.com
Mon Apr 6 08:17:08 PDT 2009
On or about 2009 Apr 6, at 9:21 AM, Olemis Lang indited:
> - The recent thread about higlighting the tests that were previously
> run but didnt show up at testing time ... this is not precisely an
> error, but perhaps means that something weird was going on, or
> maybe not ;).
Perhaps, but I don't think that should be part of the regression
> - The same use case we are talking about ... if setUp fails there are
> mainly three options to consider :
> * signal a failed test case ... but IMHO this *SHOULD NOT* be used
> since the test itself didnot fail ... it just couldnt be
> performed ... so a better approach is ...
Agreed that is not an option.
> * ... to signal a test failure, indicating that the test code was
> not accurate and needs to be fixed. But maybe this is not the
> case. Perhaps under certain circumstances the lack of resources
> is not a test failure at all, but a consequence of a specific
> config ... and in this case maybe it is better ...
Yes, and we use "Skip" in this case, though in our version of the
unittest framework we use an eligibility test before even getting to
setUp, and that catches most of these "skip in setUp" kinds of cases.
> * ... to log a warning message ... ;)
One other thing we do in setUp is protect our code with asserts to
indicate that we need to go back and write more test code. For
example, if the product specs says: "Device can do X", we need to test
X. However, if the current hardware can't do X, we assert that to
protect our test from producing an false-postive/PASS. Sure, we could
speculatively write code to test X, but if we can't run that code, it
is highly suspect and likely to require bug fixes when X is actually
made available. Instead of wasting time on code we can't run, we
protect our tests with asserts. Our philosophy over all is that a test
FAILure is a bug in the device-under-test. A test ERRor is a bug in
the test itself, not the device.
More information about the testing-in-python