<br><br><div class="gmail_quote">On Wed, Jan 20, 2010 at 9:12 AM, Olemis Lang <span dir="ltr"><<a href="mailto:olemis@gmail.com">olemis@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On Wed, Jan 20, 2010 at 5:42 AM, holger krekel <<a href="mailto:holger@merlinux.eu">holger@merlinux.eu</a>> wrote:<br>
> On Tue, Jan 19, 2010 at 15:35 -0500, Olemis Lang wrote:<br>
>> On Tue, Jan 19, 2010 at 3:26 PM, Alfredo Deza <<a href="mailto:alfredodeza@gmail.com">alfredodeza@gmail.com</a>> wrote:<br>
>> > Hi<br>
>> ><br>
>> > I am aware that test methods are executed in alphabetical order.<br>
>> ><br>
>> > What would be the ideal/preferable way of dealing with methods that depend<br>
>> > in the order they are being executed?<br>
>> ><br>
>> > Given that:<br>
>> ><br>
>> >> class TestFiles(unittest.TestCase):<br>
>> >><br>
>> >> def test_second_file(self):<br>
>> >> """I depend on z_file"""<br>
>> >><br>
>> >> def test_z_file(self):<br>
>> >> """I should run before second_file"""<br>
>> ><br>
>> ><br>
>> > Would it be OK to alter the naming in the methods to make sure they are run<br>
>> > in the order I need to? Even if the naming gets really weird like:<br>
>> ><br>
>><br>
>> Very bad practice ! Dependencies between test cases can lead to<br>
>> fragile tests . TCs should be as isolated as possible<br>
>> ;o)<br>
><br>
> Some way of telling a test runner to consider tests in a certain<br>
> order makes sense to me, though.<br></div></blockquote><div><br>Can someone give me a good explanation *why* test methods are executed alphabetically rather than the normal <br>Python way (top down). ?<br><br><br> <br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
><br>
> In fact, I had user requests and own thoughts to allow declaring dependencies -<br>
> such that if a test fails subsequent ones would not be run anymore.<br>
<br>
</div>Probably (I don't know the exact details ;o), but if the first test<br>
fails that (should ?) means that something is not right (i.e. behaving<br>
like expected) with the SUT . If tester does not want to run the other<br>
tests thats because some assertions about the SUT have not been met<br>
rather than «test case X failed». I strongly suggest to write test<br>
cases by focusing on the SUT, and avoid dependencies between test<br>
(cases) . If the condition to skip the dependent test cases is exactly<br>
the same that will be asserted then I (a notoriously non-practical<br>
person that wants to sleep quietly one night after another) suggest to<br>
be explicit and create a test stub (function ? but not a test case)<br>
and write an assertion that just calls the test stub. OTOH if the<br>
framework has a skip support then use the same condition to skip the<br>
other test cases .<br>
<br>
Of course nothing like this will work with Py 2.7 skip functions since<br>
the skip decorators evaluate the condition at the wrong (IMO) time<br>
[1]_ .<br>
<br>
There are also exceptions to the rule. For example, `dutest` [2]_ maps<br>
a test case to each interactive example executed (instead of DocTest)<br>
. In that case someone (/me included) might say that when using<br>
REPORT_ONLY_FIRST_FAILURE there are some dependencies between TCs .<br>
But as you can see that's a very particular case and is conditioned by<br>
the characteristics of doctest. But XUnit-style is different and<br>
should not suffer with those symptoms.<br>
<br>
Anyway : be pragmatic ! ... and be aware of the consequences ;o)<br>
<div class="im"><br>
> py.test<br>
> runs tests in the order in which they appear in the file and i am considering<br>
> marking a test case/class as being "incremental", i.e. skip/fail tests after<br>
> the first tests fails.<br>
><br></div></blockquote><div><br>Even worse, why (if alphabetical is the correct way) test frameworks like py.test <br>do them in a different way? This adds a whole layer of confusion to someone that <br>wants to get started correctly. However I do agree that it is good to have options <br>
and different ways of doing things. But I also support conventions.<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
<br>
</div>Good luck !<br>
<br>
.. [1] Assessment of unittest 2.7 API : new features and opinions<br>
(<a href="http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html" target="_blank">http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html</a>)<br>
<br>
.. [2] dutest @ PyPI<br>
(<a href="http://pypi.python.org/pypi/dutest/0.2.3" target="_blank">http://pypi.python.org/pypi/dutest/0.2.3</a>)<br>
<div class="im"><br>
--<br>
Regards,<br>
<br>
Olemis.<br>
<br>
Blog ES: <a href="http://simelo-es.blogspot.com/" target="_blank">http://simelo-es.blogspot.com/</a><br>
Blog EN: <a href="http://simelo-en.blogspot.com/" target="_blank">http://simelo-en.blogspot.com/</a><br>
<br>
Featured article:<br>
</div>Removing PyOOP test modules. -<br>
<a href="http://flioops.hg.sourceforge.net/hgweb/flioops/dutest/rev/acc55ed8a1c0" target="_blank">http://flioops.hg.sourceforge.net/hgweb/flioops/dutest/rev/acc55ed8a1c0</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>Alfredo Deza<br><br>