[TIP] including (or not) tests within your package

[Ben Finney]

Fernando Perez <fperez.net at gmail.com> writes:

> We *always* ship the tests with all projects I am involved with, and
> work very hard to provide really easy ways for users to run the full
> test suite and provide us with useful feedback when something goes
> wrong.

I assume this means your automated tests don't have dependencies
additional to that of the non-test parts of your code.

That's a pity, because it means a lot of good testing frameworks must be
ignored, or re-implemented, or (worst of all!) installed along with your
code as another potentially-divergent copy of the same framework.

> So *my* policy is pretty simple: yes, absolutely, unconditionally,
> always, ship the tests to our users, and make sure the first thing
> they do before they ask for help or report problems is run the test
> suite.

This would involve getting users to install extra dependencies (e.g.
‘nose’, ‘setuptools’, ‘test-scenarios’, etc.) for most of my test stuff,
which makes it rather onerous to expect that to be the first thing users
do on encountering a problem.

> I've put *enormous* amounts of time and effort into making the test
> suite trivial to run for users so that this policy is viable, and I'm
> sure I could do better, but it's a start.

On the other hand, I've made use of the good testing and development
tools out there, but don't require them to be installed just to run the
application or use the library; and don't see why users would be
interested in installing all those just to submit a bug report.

-- 
 \          “One time a cop pulled me over for running a stop sign. He |
  `\        said, ‘Didn't you see the stop sign?’ I said, ‘Yeah, but I |
_o__)                don't believe everything I read.’” —Steven Wright |
Ben Finney