[TIP] alphabetical order in testing
sienkiew at stsci.edu
Wed Jan 20 09:43:13 PST 2010
Alfredo Deza wrote:
> Can someone give me a good explanation *why* test methods are executed
> alphabetically rather than the normal
> Python way (top down). ?
It's hard to say why a programmer chose a particular implementation, but
one advantage of alphabetical order is that we know what the order is.
If I have a file containing test_a, test_b, and test_c, we might
reasonably ask "What order will the tests execute when we add test_d?"
If the tests are executed in the order a for loop would find them in a
dictionary, the answer is "I don't know". If the tests are executed
alphabetically, we know that test_d will be last.
Of course, it could be any other arbitrary order, but alphabetical is
much easier to determine than "order that they appear in the file" after
you import a module.
For all that many posters have claimed that order of tests should not
matter, it does -- even for unit tests. You can _intend_ to implement
your tests so that it does not matter which order they execute in, but
what if you make a mistake and create a side-effect that influences the
outcome of a later test? In that case, it is useful to have a fixed
order to make it easier to debug your tests.
And to get back to the original question:
> Would it be OK to alter the naming in the methods to make sure they
> are run
> in the order I need to? Even if the naming gets really weird like:
As with most programming questions, the answer is "Yes, it is OK, but be
sure that you understand the implications of what you are doing".
Many posters have told you that the answer is "No". Ignore the dogmatic
statements like "Never!", but read carefully the reasons that they
give. If your tests depend on being run in a certain order, you may
cause difficulty for yourself later. The question is whether that is a
necessary difficulty or not. You should not fear complexity if you
really need it.
More information about the testing-in-python