<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#ffffff">
On 20/01/2010 14:27, Alfredo Deza wrote:
<blockquote
cite="mid:212f9b21001200627q69d93473yf82ab036bc956fe7@mail.gmail.com"
type="cite"><br>
<br>
<div class="gmail_quote">On Wed, Jan 20, 2010 at 9:12 AM, Olemis Lang
<span dir="ltr"><<a moz-do-not-send="true"
href="mailto:olemis@gmail.com">olemis@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On Wed, Jan 20, 2010 at 5:42 AM, holger krekel <<a
moz-do-not-send="true" href="mailto:holger@merlinux.eu">holger@merlinux.eu</a>>
wrote:<br>
> On Tue, Jan 19, 2010 at 15:35 -0500, Olemis Lang wrote:<br>
>> On Tue, Jan 19, 2010 at 3:26 PM, Alfredo Deza <<a
moz-do-not-send="true" href="mailto:alfredodeza@gmail.com">alfredodeza@gmail.com</a>>
wrote:<br>
>> > Hi<br>
>> ><br>
>> > I am aware that test methods are executed in alphabetical
order.<br>
>> ><br>
>> > What would be the ideal/preferable way of dealing with
methods that depend<br>
>> > in the order they are being executed?<br>
>> ><br>
>> > Given that:<br>
>> ><br>
>> >> class TestFiles(unittest.TestCase):<br>
>> >><br>
>> >> def test_second_file(self):<br>
>> >> """I depend on z_file"""<br>
>> >><br>
>> >> def test_z_file(self):<br>
>> >> """I should run before second_file"""<br>
>> ><br>
>> ><br>
>> > Would it be OK to alter the naming in the methods to make
sure they are run<br>
>> > in the order I need to? Even if the naming gets really
weird like:<br>
>> ><br>
>><br>
>> Very bad practice ! Dependencies between test cases can lead to<br>
>> fragile tests . TCs should be as isolated as possible<br>
>> ;o)<br>
><br>
> Some way of telling a test runner to consider tests in a certain<br>
> order makes sense to me, though.<br>
</div>
</blockquote>
<div><br>
Can someone give me a good explanation *why* test methods are executed
alphabetically rather than the normal <br>
Python way (top down). ?<br>
<br>
</div>
</div>
</blockquote>
<br>
How is that 'normal' in Python? Methods in a class are not ordered in
any way. In fact they are stored in a dictionary which is specifically
unordered. <br>
<br>
<br>
<blockquote
cite="mid:212f9b21001200627q69d93473yf82ab036bc956fe7@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div><br>
<br>
</div>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">><br>
> In fact, I had user requests and own thoughts to allow declaring
dependencies -<br>
> such that if a test fails subsequent ones would not be run anymore.<br>
<br>
</div>
Probably (I don't know the exact details ;o), but if the first test<br>
fails that (should ?) means that something is not right (i.e. behaving<br>
like expected) with the SUT . If tester does not want to run the other<br>
tests thats because some assertions about the SUT have not been met<br>
rather than «test case X failed». I strongly suggest to write test<br>
cases by focusing on the SUT, and avoid dependencies between test<br>
(cases) . If the condition to skip the dependent test cases is exactly<br>
the same that will be asserted then I (a notoriously non-practical<br>
person that wants to sleep quietly one night after another) suggest to<br>
be explicit and create a test stub (function ? but not a test case)<br>
and write an assertion that just calls the test stub. OTOH if the<br>
framework has a skip support then use the same condition to skip the<br>
other test cases .<br>
<br>
Of course nothing like this will work with Py 2.7 skip functions since<br>
the skip decorators evaluate the condition at the wrong (IMO) time<br>
[1]_ .<br>
<br>
There are also exceptions to the rule. For example, `dutest` [2]_ maps<br>
a test case to each interactive example executed (instead of DocTest)<br>
. In that case someone (/me included) might say that when using<br>
REPORT_ONLY_FIRST_FAILURE there are some dependencies between TCs .<br>
But as you can see that's a very particular case and is conditioned by<br>
the characteristics of doctest. But XUnit-style is different and<br>
should not suffer with those symptoms.<br>
<br>
Anyway : be pragmatic ! ... and be aware of the consequences ;o)<br>
<div class="im"><br>
> py.test<br>
> runs tests in the order in which they appear in the file and i am
considering<br>
> marking a test case/class as being "incremental", i.e. skip/fail
tests after<br>
> the first tests fails.<br>
><br>
</div>
</blockquote>
<div><br>
Even worse, why (if alphabetical is the correct way) test frameworks
like py.test <br>
do them in a different way? This adds a whole layer of confusion to
someone that <br>
wants to get started correctly. However I do agree that it is good to
have options <br>
and different ways of doing things. But I also support conventions.<br>
</div>
</div>
</blockquote>
<br>
It doesn't matter what the default order of test runs are because it is
*well understood* that individual tests need to be independent of one
another.<br>
<br>
Michael Foord<br>
<br>
<br>
<br>
<blockquote
cite="mid:212f9b21001200627q69d93473yf82ab036bc956fe7@mail.gmail.com"
type="cite">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote"
style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im"><br>
</div>
Good luck !<br>
<br>
.. [1] Assessment of unittest 2.7 API : new features and opinions<br>
(<a moz-do-not-send="true"
href="http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html"
target="_blank">http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html</a>)<br>
<br>
.. [2] dutest @ PyPI<br>
(<a moz-do-not-send="true"
href="http://pypi.python.org/pypi/dutest/0.2.3" target="_blank">http://pypi.python.org/pypi/dutest/0.2.3</a>)<br>
<div class="im"><br>
--<br>
Regards,<br>
<br>
Olemis.<br>
<br>
Blog ES: <a moz-do-not-send="true"
href="http://simelo-es.blogspot.com/" target="_blank">http://simelo-es.blogspot.com/</a><br>
Blog EN: <a moz-do-not-send="true"
href="http://simelo-en.blogspot.com/" target="_blank">http://simelo-en.blogspot.com/</a><br>
<br>
Featured article:<br>
</div>
Removing PyOOP test modules. -<br>
<a moz-do-not-send="true"
href="http://flioops.hg.sourceforge.net/hgweb/flioops/dutest/rev/acc55ed8a1c0"
target="_blank">http://flioops.hg.sourceforge.net/hgweb/flioops/dutest/rev/acc55ed8a1c0</a><br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
Alfredo Deza<br>
<br>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
testing-in-python mailing list
<a class="moz-txt-link-abbreviated" href="mailto:testing-in-python@lists.idyll.org">testing-in-python@lists.idyll.org</a>
<a class="moz-txt-link-freetext" href="http://lists.idyll.org/listinfo/testing-in-python">http://lists.idyll.org/listinfo/testing-in-python</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.ironpythoninaction.com/">http://www.ironpythoninaction.com/</a>
<a class="moz-txt-link-freetext" href="http://www.voidspace.org.uk/blog">http://www.voidspace.org.uk/blog</a>
READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
</pre>
</body>
</html>