[TIP] why you should distribute tests with your application / module

[Jean-Paul Calderone]

On Wed, 17 Sep 2008 09:59:15 +1000, Michael Gratton <michael at quuxo.com> wrote:
>
>Hi Kumar,
>
>On Tue, 2008-09-16 at 13:50 -0500, Kumar McMillan wrote:
>> What do you lose by distributing tests with your package?  10k of disk
>> space?  I can't think of anything else.
>
>With a source distribution? Absolutely - if you're giving people the
>source and a way to build it then shipping the tests is vital. If you're
>giving people the ability to hack on your code, then not including tests
>disadvantages both them (they don't get to test their changes) and you
>(if they contribute back to the project, it might be buggy).
>
>However there is no point if you're shipping a "binary" distribution.
>While this seems an odd distinction to make for Python programs,
>distributing it as eggs, in deb/rpms or compiled to an exe effectively
>are binary. In these scenarios people who download them clearly aren't
>interested in modifying them, so including the tests here is pointless.

For eggs, debs, and rpms, there's absolutely a point.  Modification doesn't
really enter into the question.  Distributing the tests lets the person who
downloads the software to run them and see if it works.  Why wouldn't users
want to know if the software works?  Better still, if tests fail, then when
users report bugs, you can ask them which tests fail (or include some fancy
thing that automatically creates bug report templates including the failing
test information).

You also want to know if you managed to package the software correctly and
what better way for you to do this than to be able to run the the packaged
version of the tests?

I suspect similar arguments apply to exes, but not having worked with them
in Python very much, I can't say for sure.

Jean-Paul