[TIP] "midget" pony continuous integration tool

Ben Finney ben+python at benfinney.id.au
Wed May 6 21:44:05 PDT 2009


Laura Creighton <lac at openend.se> writes:

> In a message of Thu, 07 May 2009 10:45:24 +1000, Ben Finney writes:
> >So, if I make a change to fix one of the failures and inadvertantly
> >break a test that previously succeeded, I can go an indeterminate
> >number of code-test cycles before I find out about it?
> >
> >That seems like the canonical reason why “run the entire unit test
> >suite every time� is good practice, and I can't see why the
> >above behaviour would be desirable.
> 
> Because running all the tests takes 2.5 hours.

Note I'm talking specifically about running the *unit test* suite (only)
after every interesting change. If your unit tests take hours to run
rather than seconds, you've got insufficient isolation of the code units
during testing.

If, on the other hand, you're talking about tests wider in scope than
unit tests, then certainly I'm not saying they need to be run on every
change. The point of tight, isolated unit tests is precisely that they
are fast and focussed, as contrasted with other types of test.

-- 
 \       “Faith, n. Belief without evidence in what is told by one who |
  `\   speaks without knowledge, of things without parallel.” —Ambrose |
_o__)                           Bierce, _The Devil's Dictionary_, 1906 |
Ben Finney




More information about the testing-in-python mailing list