[TIP] why you should distribute tests with your application / module

[C. Titus Brown]

On Thu, Sep 18, 2008 at 10:21:12AM +1000, Michael Gratton wrote:
-> On Wed, 2008-09-17 at 07:55 -0700, C. Titus Brown wrote:
-> > Tests are also useful to see if programs run on other platforms, and
-> > they can give the developer debugging feedback that they can't obtain
-> > otherwise.
-> 
-> Sure, but in such a case you would need to obtain a source distribution
-> anyway - if only to recompile any native code (or bindings to native
-> code) it might contain or to produce a egg for that platform.
-> 
-> >   For example, I do most of my development on Linux, and with
-> > good test coverage I can run my tests on Windows and Mac OS X to see if
-> > things break.  Even if I didn't have access to those platforms as a
-> > developer, someone who *does* could run my tests and see if things
-> > worked.
-> 
-> Yes, see above.
-> 
-> End users use binary distributions and don't use or care about tests.
-> Developers, porters and packagers use source distributions and do care
-> about tests. So, ship them with your source, but don't bother with your
-> binaries.
-> 
-> Or, to look at it another way, if your users need to run your tests, you
-> (or your linux distro) have already shipped a broken binary.
-> 
-> Mozilla has a large suite of test cases. Do you wish they shipped those
-> so you could run them every time you download Firefox? If they did,
-> would you?

Win 95, Win 98, Win NT, Win XP, Win 2000, Vista.  There are binaries
that will work on all of them, and binaries that will work on only one
of them.

Mac OS X 10.0, 10.1, 10.2, 10.3, 10.4, 10.5, ditto.

If I were a Firefox developer, I would want to be able to tell users
getting weird errors to run the tests, yes.

(I do ship a binary for Mac OS X and Windows, and it does sometimes
fail, and no, I don't ship it with a test suite.  But I should :)

--titus
-- 
C. Titus Brown, ctb at msu.edu