[TIP] tox + py.test: test installed sdist

[Alexander Steinert]

Dear list,

there one thing in tox I don't quite understand yet. Even though tox creates
the sdist and installs it into the virtual env, the installed version is not
used/tested if tox is started in the project's root and the source of the
module/package to be tested lives there too -- which should be quite common.

Testing the version in the source tree would hide potential packaging errors.
I would like to test the *installed* version. The directory with the installed
version is last in sys.path! Am I assumed to hack sys.path?

Minimal example
###############

$ hg manifest
.hgignore
module.py
py-1.4.3.zip
pytest-2.0.3.zip
setup.py
test.py
tox.ini

$ cat setup.py
from distutils.core import setup
setup(
    name = "module",
    version = "0.1",
    py_modules = ["module"],
    )

$ cat module.py
member = 42

$ cat test.py
import sys
from os import path
from pprint import pprint
import module
def test_member():
    assert module.member == 42
def test_path():
    pprint(sys.path)
    pprint(module)
    expected =  ".tox/py26/lib/python2.6/site-packages/module.pyc"
    assert path.exists(expected)
    actual = path.relpath(module.__file__)
    assert actual == expected

$ cat tox.ini
[testenv]
commands = py.test --verbose test.py
deps =
    {toxinidir}/py-1.4.3.zip
    {toxinidir}/pytest-2.0.3.zip

$ tox --recreate -e py26
_________________________________ [tox sdist] __________________________________
[TOX] ***creating sdist package
[TOX] /home/steinale/work/toxtests$ /usr/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
[TOX] ***copying new sdistfile to '/home/steinale/.tox/distshare/module-0.1.zip'
______________________________ [tox testenv:py26] ______________________________
[TOX] ***recreating virtualenv py26 (configchange/incomplete install detected)
[TOX] /home/steinale/work/toxtests/.tox$ /usr/local/bin/virtualenv --distribute --no-site-packages -p /usr/bin/python2.6 py26 >py26/log/0.log
[TOX] ***installing dependencies: /home/steinale/work/toxtests/py-1.4.3.zip, /home/steinale/work/toxtests/pytest-2.0.3.zip
[TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download /home/steinale/work/toxtests/py-1.4.3.zip /home/steinale/work/toxtests/pytest-2.0.3.zip >1.log
[TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download ../../dist/module-0.1.zip >2.log
[TOX] /home/steinale/work/toxtests$ .tox/py26/bin/py.test --verbose test.py
============================= test session starts ==============================
platform linux2 -- Python 2.6.5 -- pytest-2.0.3 -- /home/steinale/work/toxtests/.tox/py26/bin/python2.6
collecting ... collected 2 items

test.py:5: test_member PASSED
test.py:7: test_path FAILED

=================================== FAILURES ===================================
__________________________________ test_path ___________________________________

    def test_path():
        pprint(sys.path)
        pprint(module)
        expected =  ".tox/py26/lib/python2.6/site-packages/module.pyc"
        assert path.exists(expected)
        actual = path.relpath(module.__file__)
>       assert actual == expected
E       assert 'module.pyc' == '.tox/py26/lib/python2....te-packages/module.pyc'
E         - module.pyc
E         + .tox/py26/lib/python2.6/site-packages/module.pyc

test.py:13: AssertionError
------------------------------- Captured stdout --------------------------------
['/home/steinale/work/toxtests',
 '/home/steinale/work/toxtests/.tox/py26/bin',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages/distribute-0.6.19-py2.6.egg',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages/pip-1.0.1-py2.6.egg',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/plat-linux2',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-tk',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-old',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-dynload',
 '/usr/lib/python2.6',
 '/usr/lib/python2.6/plat-linux2',
 '/usr/lib/python2.6/lib-tk',
 '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages']
<module 'module' from '/home/steinale/work/toxtests/module.pyc'>
====================== 1 failed, 1 passed in 0.03 seconds ======================
[TOX] ERROR: InvocationError: '.tox/py26/bin/py.test --verbose test.py'
________________________________ [tox summary] _________________________________
[TOX] ERROR: py26: commands failed