[TIP] alphabetical order in testing

[Mark Sienkiewicz]

Alfredo Deza wrote:
> Can someone give me a good explanation *why* test methods are executed
> alphabetically  rather than the normal
> Python way (top down). ?
>
>   

It's hard to say why a programmer chose a particular implementation, but 
one advantage of alphabetical order is that we know what the order is.

If I have a file containing test_a, test_b, and test_c, we might 
reasonably ask "What order will the tests execute when we add test_d?"  
If the tests are executed in the order a for loop would find them in a 
dictionary, the answer is "I don't know".  If the tests are executed 
alphabetically, we know that test_d will be last.

Of course, it could be any other arbitrary order, but alphabetical is 
much easier to determine than "order that they appear in the file" after 
you import a module.

For all that many posters have claimed that order of tests should not 
matter, it does -- even for unit tests.  You can _intend_ to implement 
your tests so that it does not matter which order they execute in, but 
what if you make a mistake and create a side-effect that influences the 
outcome of a later test?  In that case, it is useful to have a fixed 
order to make it easier to debug your tests.


And to get back to the original question:

> Would it be OK to alter the naming in the methods to make sure they 
> are run
> in the order I need to? Even if the naming gets really weird like:

As with most programming questions, the answer is "Yes, it is OK, but be 
sure that you understand the implications of what you are doing".

Many posters have told you that the answer is "No".  Ignore the dogmatic 
statements like "Never!", but read carefully the reasons that they 
give.  If your tests depend on being run in a certain order, you may 
cause difficulty for yourself later.  The question is whether that is a 
necessary difficulty or not.  You should not fear complexity if you 
really need it.

Mark S.