[TIP] alphabetical order in testing

C. Titus Brown ctb at msu.edu
Wed Jan 20 07:32:20 PST 2010


On Wed, Jan 20, 2010 at 10:26:38AM -0500, Olemis Lang wrote:
> On Wed, Jan 20, 2010 at 10:17 AM, Olemis Lang <olemis at gmail.com> wrote:
> > On Wed, Jan 20, 2010 at 9:52 AM, C. Titus Brown <ctb at msu.edu> wrote:
> >> On Wed, Jan 20, 2010 at 09:12:29AM -0500, Olemis Lang wrote:
> >>> On Wed, Jan 20, 2010 at 5:42 AM, holger krekel <holger at merlinux.eu> wrote:
> >>> > Some way of telling a test runner to consider tests in a certain
> >>> > order makes sense to me, though.
> >>> >
> >>> > In fact, I had user requests and own thoughts to allow declaring dependencies -
> >>> > such that if a test fails subsequent ones would not be run anymore.
> >>>
> >>> Probably (I don't know the exact details ;o), but if the first test
> >>> fails that (should ?) means that something is not right (i.e. behaving
> >>> like expected) with the SUT .
> >>
> >> Olemis, et al. -- independence of test order is generally considered a
> >> requirement for unit tests, but not for other tests (functional, UI, etc.)
> >> where it simply wouldn't make sense.
> 
> BTW, D'u have some (concrete ?) examples illustrating this ?

A simple thought experiment is sufficient, really (which I like because
it asks people to think rather than arguing that your specific example
doesn't match their situation :)

For example, consider:

Anywhere where a long series of actions needs to be tested (typical
functional testing of an execution path through a Web site, for example).
The goal is to test all of the actions together, either for acceptance
test purposes, or pre- and post-condition testing, or ... Breaking this
up into a set of unit tests would be testing something different altogether.

Suppose you have a 5gb database that you are testing for performance and
correctness in querying.  Multiple independent set up and teardown events would
be nonsensical in this case.

Automated GUI testing is another situation where the unit test setup/teardown
approach *can* work but need not always apply.

Integration tests, where the goal is to test the system as a whole -- almost
the diametric opposite of unit tests -- are rarely going to be order
independent.

Grig, any other examples?

Unit tests are a small -- important, but small -- subset of the testing world
:)

--titus
-- 
C. Titus Brown, ctb at msu.edu



More information about the testing-in-python mailing list