[TIP] including (or not) tests within your package

[Olemis Lang]

On Thu, Jul 29, 2010 at 10:10 AM, Barry Warsaw <barry at python.org> wrote:
> On Jul 28, 2010, at 07:36 AM, Olemis Lang wrote:
>
>>This is the typical situation of using tests_require (testing deps) vs
>>install_requires (pkg deps) in setuptools . Packages listed using the
>>former option are not installed after executing `install` command .
>>This only happens if `test` command is executed and thay are not
>>already installed ;o)
>
> Sure, the issue of additional dependencies getting pulled in just to run the
> tests, can be an important consideration.  I'm personally (in the typical
> case) okay with just installing those dependencies, but I get that there will
> be exceptions, either because end-users don't want to pull in those
> dependencies or for large packages, they can be really numerous or expensive.
>

Probably I'm missing something .

  - If advocating for *always* including test code in packages , it's
    'cause they'll be useful somehow at a given time (isn't it ? otherwise
    there's no point at all for including tests in there)
  - If this is correct , then all *those dependencies* and|or
    *large packages* (probably) *really numerous or expensive*
    will be pulled in
  - Otherwise, if tests will be there and will not be run then :
    Why would anyone want to include them in a production system ?
    I wouldn't like either people reviewing and playing with artifacts
    deployed in there just to figure out how the system works or ...
  - If «end-users don't want to pull in those dependencies» there are
    three or four ways to go
    1. declare *critical* testing deps (i.e. if missing can't test anything)
       in `tests_require` . Those will be installed only if `test` command
       is invoked (so it's obvious & explicit the fact that end-user wants
       to test the pkg ;o), and either if the user doesn't want to install'em
       or they cannot be installed (e.g. no way to retrieve dependent
       pkgs ) , in both cases test run will be aborted (see assumption
       above )
    2. don't declare *optional* testing deps anywhere and prepare test
       code to skip test cases that depend on them (e.g. that's what
       happened in Trac's XmlRpcPlugin, prior to version 1.1.0 some
       test cases that needed twill were «skipped» -not loaded- ,
       AFAICR now option 1 is used and the plugin works without
       `twill` , which is retrieved inly if `test` command is invoked ;o)
    3. Damn !!! I'm old . I forgot the other one or two while I was writing
        the first two ... :-$

Probably I'm missing something .

> I wonder if it will end up kind of like the documentation case.  More
> dependencies can get pulled to build the docs (e.g. Sphinx) but after a while
> you've pretty much installed everything you're going to care about
> (e.g. jinja2 only gets installed once ;).

Probably I'm missing something .

Let's say that the package is installed in , let's say, enterprise X
to solve problem Y . Are you suggesting that it's ok to install
Sphinx, docutils, latex, ... in each and every PC and generate docs
once for each workstation | server ? Rather than this I'd use another
PC so as to build docs once and prepare (one or two) packages ready to
be deployed (installed) in the end users' PCs. There's where I see the
difference between dev workstations (sdists, test suites, auto-doc
libs, ...) & production sites (essentially just the software &
probably user guides & similar docs)

> After that, your cache is nice and
> warm and installing a new package with its test dependencies might not
> actually end up installing anything new.
>

I don't understand , if it's already installed what's the problem ? If
it's not installed and is required to do what you want (testing in
this particular case) then you might want to install it . Otherwise
you don't really want to do what you want to do (or just can't do it,
which is a valid reason too, considering that all obstacles are there
because it was your choice ;o).

PS: When I say *YOU* I'm think about the hypothetical end-user .

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Soluciones Olimpiadas de Matemática IMO 2007 - Problema 4 -
http://soluciones-olimpiadas.blogspot.com/2010/07/imo-2007-problema-4.html