[TIP] unittest q: keeping track of the *right* number of tests

[laidler at stsci.edu]

>
>On Sun, Mar 22, 2009 at 4:15 PM, C. Titus Brown <ctb at msu.edu> wrote:
>> Hi all,
>>
>> we're running into an interesting problem over on the pygr project: the
>> number of tests to run varies according to the installation, and we're
>> having trouble tracking the expected number of tests.
>>
>> Briefly, we include or skip entire test suites depending on whether or
>> not MySQL, sqlite, and/or BLAST (a sequence comparison package) are
>> installed.  How should we track how many tests *should* actually be run?
>>
>> (This is a more general problem, incidentally; I've noticed that in
>> projects with thousands of tests, a few can slip through the cracks and
>> become "not executed" without anyone noticing, for a variety of reasons
>> (inappropriate skipping, poor if statement usage, function renaming,
>> etc.)

The solution we have adopted is to build this into a separate
test report system, which remembers the names of all the tests
that have previously been run in a given environment and 
reports any missing tests.

Vicki Laidler