[TIP] including (or not) tests within your package

[Ben Finney]

Andrew Bennetts <andrew at bemusement.org> writes:

> Ben Finney wrote:
> > [One downside of installing the tests along with the rest of the
> > package is that] They would thereby become part of the *public*
> > interface for the package. I don't want that implied promise; the
> > test suite is subject to arbitrary unannounced change or removal.
>
> Surely you don't consider every importable name to be part of the
> public interface for a package?

Every importable name that isn't marked with a leading-underscore name,
yes. That's what that distinction is for, after all.

> If implied publicness of the API is's your concern, call your tests
> subpackage “_tests”. Or put “NOT PUBLIC API; DO NOT USE” at the top of
> the docstring for that package, or whatever convention you usually use
> to signal to your API clients which parts aren't public.

Or, even cleaner, don't install them at all as part of the package
installation.

> > They're for use while developing (hence part of the source), not in
> > their deployed location (hence *not* part of the binary package).
>
> What about packages that build on your packages?

The test code is still available in the source distribution. What use
case are you thinking of?

> Do you provide a foolib-tests (or maybe foolib-dev) binary package
> alongside the foolib package?  Or provide a public foolib.test_support
> module that has all the bits you expect others to need?   [They both
> seem like plausible answers to me, but I'm curious to hear if someone
> has experience with these approaches.]

Neither. The test cases are available and useable in the source
distribution, which someone doing development using this package will
need anyway.

> Also, there is some value to enabling users to run tests in their
> deployed environment, where they may have a unique combination of
> dependency versions, system configuration, etc.

I can see that, and I would like finer-grained control to be available
to users. But they do have the ‘python ./setup.py test’ command
available, at least with Setuptools and Distribute and friends. I don't
see the pollution of the system namespace to be worth it.

-- 
 \        “All opinions are not equal. Some are a very great deal more |
  `\    robust, sophisticated and well supported in logic and argument |
_o__)                                     than others.” —Douglas Adams |
Ben Finney