[TIP] unittest q: keeping track of the *right* number of tests

[C. Titus Brown]

On Mon, Mar 23, 2009 at 09:47:14AM +1100, Robert Collins wrote:
-> On Sun, 2009-03-22 at 07:15 -0700, C. Titus Brown wrote:
-> > Hi all,
-> > 
-> > we're running into an interesting problem over on the pygr project: the
-> > number of tests to run varies according to the installation, and we're
-> > having trouble tracking the expected number of tests.
-> 
-> In bzr we track the total and track non-executed tests by cause:
-> Missing feature 'Internally performed glob expansion' skipped 1 tests.
-> Missing feature 'bzrlib._dirstate_helpers_c' skipped 2 tests.
-> Missing feature 'bzrlib._readdir_pyx' skipped 1 tests.
-> 
-> Its fairly easy to see if these jump around. (I ran 200 tests of our 17K
-> suite to get this quick report :)).

Yes, exactly!  But we still want to have something alert us to missing
tests loudly:

	Missing feature 'foo bar' skipped 1 tests; THERE SHOULD HAVE
	BEEN 5 OTHERS.  DIE DIE DIE *** LOOK HERE YOU FOOL ***

If Michael Foord and others don't know of a standard unittest extension
to do this, then it probably doesn't exist!  I guess we'll just have to
hack something together...  I abandoned unittest a while ago, but we're
still using it in pygr so I will have to deal ;)

cheers,
--titus
-- 
C. Titus Brown, ctb at msu.edu