[TIP] OT: mailing list netiquette (Re: alphabetical order in testing)

C. Titus Brown ctb at msu.edu
Wed Jan 20 06:54:54 PST 2010


<mailing list moderator mode>

Please remember to quote appropriately.  Some of us are using '90s technology
to read our e-mail.

And avoid top posting at all costs.

</mailing list moderator mode>

On Wed, Jan 20, 2010 at 09:34:41AM -0500, Alfredo Deza wrote:
> On Wed, Jan 20, 2010 at 9:32 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:
> 
> >  On 20/01/2010 14:27, Alfredo Deza wrote:
> >
> >
> >
> > On Wed, Jan 20, 2010 at 9:12 AM, Olemis Lang <olemis at gmail.com> wrote:
> >
> >> On Wed, Jan 20, 2010 at 5:42 AM, holger krekel <holger at merlinux.eu>
> >> wrote:
> >> > On Tue, Jan 19, 2010 at 15:35 -0500, Olemis Lang wrote:
> >> >> On Tue, Jan 19, 2010 at 3:26 PM, Alfredo Deza <alfredodeza at gmail.com>
> >> wrote:
> >> >> > Hi
> >> >> >
> >> >> > I am aware that test methods are executed in alphabetical order.
> >> >> >
> >> >> > What would be the ideal/preferable way of dealing with methods that
> >> depend
> >> >> > in the order they are being executed?
> >> >> >
> >> >> > Given that:
> >> >> >
> >> >> >> class TestFiles(unittest.TestCase):
> >> >> >>
> >> >> >>     def test_second_file(self):
> >> >> >>         """I depend on z_file"""
> >> >> >>
> >> >> >>     def test_z_file(self):
> >> >> >>         """I should run before second_file"""
> >> >> >
> >> >> >
> >> >> > Would it be OK to alter the naming in the methods to make sure they
> >> are run
> >> >> > in the order I need to? Even if the naming gets really weird like:
> >> >> >
> >> >>
> >> >> Very bad practice ! Dependencies between test cases can lead to
> >> >> fragile tests . TCs should be as isolated as possible
> >> >> ;o)
> >> >
> >> > Some way of telling a test runner to consider tests in a certain
> >> > order makes sense to me, though.
> >>
> >
> > Can someone give me a good explanation *why* test methods are executed
> > alphabetically  rather than the normal
> > Python way (top down). ?
> >
> >
> > How is that 'normal' in Python? Methods in a class are not ordered in any
> > way. In fact they are stored in a dictionary which is specifically
> > unordered.
> >
> 
> Ah, correct, I was thinking functions not classes!
> 
> >
> >
> >
> >
> >
> >> >
> >> > In fact, I had user requests and own thoughts to allow declaring
> >> dependencies -
> >> > such that if a test fails subsequent ones would not be run anymore.
> >>
> >>  Probably (I don't know the exact details ;o), but if the first test
> >> fails that (should ?) means that something is not right (i.e. behaving
> >> like expected) with the SUT . If tester does not want to run the other
> >> tests thats because some assertions about the SUT have not been met
> >> rather than ?test case X failed?. I strongly suggest to write test
> >> cases by focusing on the SUT, and avoid dependencies between test
> >> (cases) . If the condition to skip the dependent test cases is exactly
> >> the same that will be asserted then I (a notoriously non-practical
> >> person that wants to sleep quietly one night after another) suggest to
> >> be explicit and create a test stub (function ? but not a test case)
> >> and write an assertion that just calls the test stub. OTOH if the
> >> framework has a skip support then use the same condition to skip the
> >> other test cases .
> >>
> >> Of course nothing like this will work with Py 2.7 skip functions since
> >> the skip decorators evaluate the condition at the wrong (IMO) time
> >> [1]_ .
> >>
> >> There are also exceptions to the rule. For example, `dutest` [2]_ maps
> >> a test case to each interactive example executed (instead of DocTest)
> >> . In that case someone (/me included) might say that when using
> >> REPORT_ONLY_FIRST_FAILURE there are some dependencies between TCs .
> >> But as you can see that's a very particular case and is conditioned by
> >> the characteristics of doctest. But XUnit-style is different and
> >> should not suffer with those symptoms.
> >>
> >> Anyway : be pragmatic ! ... and be aware of the consequences ;o)
> >>
> >> > py.test
> >> > runs tests in the order in which they appear in the file and i am
> >> considering
> >> > marking a test case/class as being "incremental", i.e. skip/fail tests
> >> after
> >> > the first tests fails.
> >> >
> >>
> >
> > Even worse, why (if alphabetical is the correct way) test frameworks like
> > py.test
> > do them in a different way? This adds a whole layer of confusion to someone
> > that
> > wants to get started correctly. However I do agree that it is good to have
> > options
> > and different ways of doing things. But I also support conventions.
> >
> >
> > It doesn't matter what the default order of test runs are because it is
> > *well understood* that individual tests need to be independent of one
> > another.
> >
> > Michael Foord
> >
> >
> >
> >
> >
> >>
> >>  Good luck !
> >>
> >> .. [1] Assessment of unittest 2.7 API : new features and opinions
> >>        (
> >> http://simelo-en.blogspot.com/2009/12/assessment-of-unittest-27-api-new.html
> >> )
> >>
> >> .. [2] dutest @ PyPI
> >>        (http://pypi.python.org/pypi/dutest/0.2.3)
> >>
> >> --
> >> Regards,
> >>
> >> Olemis.
> >>
> >> Blog ES: http://simelo-es.blogspot.com/
> >> Blog EN: http://simelo-en.blogspot.com/
> >>
> >> Featured article:
> >>  Removing PyOOP test modules.  -
> >> http://flioops.hg.sourceforge.net/hgweb/flioops/dutest/rev/acc55ed8a1c0
> >>
> >
> >
> >
> > --
> > Alfredo Deza
> >
> >
> > _______________________________________________
> > testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
> >
> >
> >
> > -- http://www.ironpythoninaction.com/http://www.voidspace.org.uk/blog
> >
> > READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (?BOGUS AGREEMENTS?) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
> >
> >
> >
> 
> 
> -- 
> Alfredo Deza

> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python


-- 
C. Titus Brown, ctb at msu.edu



More information about the testing-in-python mailing list