[TIP] tox + py.test: test installed sdist

Alexander Steinert sputnik at termo.net
Wed Jul 6 06:04:57 PDT 2011

holger krekel, Mon 2011-07-04 22:10 CEST:
> > there one thing in tox I don't quite understand yet. Even though tox
> > creates the sdist and installs it into the virtual env, the
> > installed version is not used/tested if tox is started in the
> > project's root and the source of the module/package to be tested
> > lives there too -- which should be quite common.
> thanks for the good report.  In order to be able to import the test.py
> module py.test adds the parent directory to sys.path, thus the source
> module.py is found when it is imported.  For running py.test (or
> nosetests which does the same) standalone this sys.path hackery works.
> Most people have their tests and package in subdirs.
> If you put test.py into a subdir, does it work?

No. I changed my example to be less minimal and more realistic (see
below).The result is the same. (Abs. path to) . is first in sys.path and
./.tox/py26/lib/python2.6/site-packages is last.

IMHO, tox should take care that, during tests, modules are imported from
the installed sdist. Imagine your setup.py is flaky, say you package
no/wrong data files. You would never see that when you run tox.

> FWIW I once payed with tox accepting a pattern for test files
> and copy them to a fresh directory and then run tests there.
> This would avoid the issue you are seeing i think.

Hmm, even if I use changedir=package to let tox call py.test there,
toxinidir is first in path :-(


Excerpt from the modified example:

$ hg manifest

$ cat setup.py
from distutils.core import setup
    name = "package",
    version = "0.1",
    packages = ["package"],

$ cat tox.ini
commands = py.test --verbose package/tests
deps =

$ tox --recreate -e py26
_________________________________ [tox sdist] __________________________________
[TOX] ***creating sdist package
[TOX] /home/steinale/work/toxtests$ /usr/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
[TOX] ***copying new sdistfile to '/home/steinale/.tox/distshare/package-0.1.zip'
______________________________ [tox testenv:py26] ______________________________
[TOX] ***recreating virtualenv py26 (configchange/incomplete install detected)
[TOX] /home/steinale/work/toxtests/.tox$ /usr/local/bin/virtualenv --distribute --no-site-packages -p /usr/bin/python2.6 py26 >py26/log/0.log
[TOX] ***installing dependencies: /home/steinale/work/toxtests/py-1.4.3.zip, /home/steinale/work/toxtests/pytest-2.0.3.zip
[TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download /home/steinale/work/toxtests/py-1.4.3.zip /home/steinale/work/toxtests/pytest-2.0.3.zip >1.log
[TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download ../../dist/package-0.1.zip >2.log
[TOX] /home/steinale/work/toxtests$ .tox/py26/bin/py.test --verbose package/tests
============================= test session starts ==============================
platform linux2 -- Python 2.6.5 -- pytest-2.0.3 -- /home/steinale/work/toxtests/.tox/py26/bin/python2.6
collecting ... collected 2 items

package/tests/test_module.py:5: test_member PASSED
package/tests/test_module.py:7: test_path FAILED

=================================== FAILURES ===================================
__________________________________ test_path ___________________________________

    def test_path():
        expected =  ".tox/py26/lib/python2.6/site-packages/package/module.pyc"
        assert path.exists(expected)
        actual = path.relpath(module.__file__)
>       assert actual == expected
E       assert 'package/module.pyc' == '.tox/py26/lib/pyth...package/module.pyc'
E         - package/module.pyc
E         + .tox/py26/lib/python2.6/site-packages/package/module.pyc

package/tests/test_module.py:13: AssertionError
------------------------------- Captured stdout --------------------------------
<module 'package.module' from '/home/steinale/work/toxtests/package/module.pyc'>
====================== 1 failed, 1 passed in 0.03 seconds ======================
[TOX] ERROR: InvocationError: '.tox/py26/bin/py.test --verbose package/tests'
________________________________ [tox summary] _________________________________
[TOX] ERROR: py26: commands failed

More information about the testing-in-python mailing list