[TIP] "midget" pony continuous integration tool

[Ben Finney]

holger krekel <holger at merlinux.eu> writes:

> On Thu, May 07, 2009 at 10:45 +1000, Ben Finney wrote:
> > So, if I make a change to fix one of the failures and inadvertantly
> > break a test that previously succeeded, I can go an indeterminate
> > number of code-test cycles before I find out about it?
> 
> well, you will all the time work on a failing test - most people i
> know like to focus on fixing one particular test and then see if that
> broke any other bits.

If I make a change that breaks an existing test that used to pass, then
I need to know about that as early as possible to find a better
solution.

That implies that all the unit tests need to run when I make that
change, not just the one I'm focussing on. The point of an automated
test suite is that I don't need to remember which of them need to run.

> > That seems like the canonical reason why ???run the entire unit test
> > suite every time??? is good practice, and I can't see why the above
> > behaviour would be desirable.
> 
> As Laura points out, if tests take longer it is nice to not have to
> wait before seeing if the intended-to-fix test works. Btw, py.test
> runs unit, functional and integration tests and i think nosetests also
> does runs more than pure unit tests.

I think you mean it *can* run all those kinds of tests; my point is that
it can *also* be more specific.

Acceptance, integration, performance, etc. tests are *not* unit tests,
and unit tests need to be executable as an isolated suite precisely
because they're the type of tests one wants to run as often as possible.

-- 
 \       “The surest way to corrupt a youth is to instruct him to hold |
  `\       in higher esteem those who think alike than those who think |
_o__)                               differently.” —Friedrich Nietzsche |
Ben Finney