[TIP] Tasting the Cheeseshop

Barry Warsaw barry at python.org
Thu Oct 20 10:22:42 PDT 2011

On Oct 17, 2011, at 10:36 PM, exarkun at twistedmatrix.com wrote:

>One thing that turns me off of this approach is that various tools have taken
>to executing setup.py in all kinds of strange contexts, so it's not at all
>clear what expectations authors should have when trying to make `python
>setup.py test` work.

I think that's up to us to define.

>What parts of the project under test will be importable?  What kind of
>environment setup is `python setup.py test` responsible for?  Is `python
>setup.py test` supposed to work on a vanilla unpacked tarball, or is it
>expected to run only after `python setup.py install` has succeeded?

All good questions.  In my experiment, the command essentially runs under a
virtualenv (using tox), so it'll download dependencies from PyPI and run the
tests under a venv install.

We were chatting about this at our recent BACONPIG meeting and it was pointed
out that system-level dependencies don't get resolved this way.  E.g. if you
depend on an extension module that depends on libjpeg-dev being installed,
this might fail.  There are also assumptions about how Python is built and
what extensions it has available.  I think we should start to define what
these assumptions are.

>What if the test runner is part of the project under test?  What if there is
>already an installed version of the project, different from the one being

Again, venv solves some of this.

>What if there are dependencies needed before the tests can be run?

See above.  Also, it will be problematic if there are manual setup steps
needed before the tests can run.  E.g. Storm's tests need to have PostgreSQL
and MySQL manually set up before its tests can run.

In some cases, running a subset of the full test suite will have to suffice,
and I think is better than *no* tests.

>If you propose what you want to work in more detail, then maybe people will
>see that it's possible for them to provide the behavior.  It seems like it
>might avoid the need to rummage through a lot of baggage to just avoid the
>setup.py tarpit though.

A question: would it be possible to get setup.py's `test_*` keys into egg info
so they can be introspected rather than having to grep setup.py?

Something I think Jason Coombs suggested, which seems kind of neat.

# ...

setup_keywords = {
    # ...
    'test_suite': 'mypkg.tests',
    'use_2to3': True,
    'convert_2to3_doctest': find_doctests(),
    # ...

if __name__ == '__main__':

This would let you `import setup` and then dig the keys out of the
setup_keywords dictionary.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20111020/737bb6c8/attachment.pgp>

More information about the testing-in-python mailing list