[TIP] alphabetical order in testing
olemis at gmail.com
Wed Jan 20 06:12:29 PST 2010
On Wed, Jan 20, 2010 at 5:42 AM, holger krekel <holger at merlinux.eu> wrote:
> On Tue, Jan 19, 2010 at 15:35 -0500, Olemis Lang wrote:
>> On Tue, Jan 19, 2010 at 3:26 PM, Alfredo Deza <alfredodeza at gmail.com> wrote:
>> > Hi
>> > I am aware that test methods are executed in alphabetical order.
>> > What would be the ideal/preferable way of dealing with methods that depend
>> > in the order they are being executed?
>> > Given that:
>> >> class TestFiles(unittest.TestCase):
>> >> def test_second_file(self):
>> >> """I depend on z_file"""
>> >> def test_z_file(self):
>> >> """I should run before second_file"""
>> > Would it be OK to alter the naming in the methods to make sure they are run
>> > in the order I need to? Even if the naming gets really weird like:
>> Very bad practice ! Dependencies between test cases can lead to
>> fragile tests . TCs should be as isolated as possible
> Some way of telling a test runner to consider tests in a certain
> order makes sense to me, though.
> In fact, I had user requests and own thoughts to allow declaring dependencies -
> such that if a test fails subsequent ones would not be run anymore.
Probably (I don't know the exact details ;o), but if the first test
fails that (should ?) means that something is not right (i.e. behaving
like expected) with the SUT . If tester does not want to run the other
tests thats because some assertions about the SUT have not been met
rather than «test case X failed». I strongly suggest to write test
cases by focusing on the SUT, and avoid dependencies between test
(cases) . If the condition to skip the dependent test cases is exactly
the same that will be asserted then I (a notoriously non-practical
person that wants to sleep quietly one night after another) suggest to
be explicit and create a test stub (function ? but not a test case)
and write an assertion that just calls the test stub. OTOH if the
framework has a skip support then use the same condition to skip the
other test cases .
Of course nothing like this will work with Py 2.7 skip functions since
the skip decorators evaluate the condition at the wrong (IMO) time
There are also exceptions to the rule. For example, `dutest` _ maps
a test case to each interactive example executed (instead of DocTest)
. In that case someone (/me included) might say that when using
REPORT_ONLY_FIRST_FAILURE there are some dependencies between TCs .
But as you can see that's a very particular case and is conditioned by
the characteristics of doctest. But XUnit-style is different and
should not suffer with those symptoms.
Anyway : be pragmatic ! ... and be aware of the consequences ;o)
> runs tests in the order in which they appear in the file and i am considering
> marking a test case/class as being "incremental", i.e. skip/fail tests after
> the first tests fails.
Good luck !
..  Assessment of unittest 2.7 API : new features and opinions
..  dutest @ PyPI
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Removing PyOOP test modules. -
More information about the testing-in-python