[TIP] tox + py.test: test installed sdist

[holger krekel]

Hi Alexander,

On Mon, Jul 04, 2011 at 15:35 +0200, Alexander Steinert wrote:
> Dear list,
> 
> there one thing in tox I don't quite understand yet. Even though tox creates
> the sdist and installs it into the virtual env, the installed version is not
> used/tested if tox is started in the project's root and the source of the
> module/package to be tested lives there too -- which should be quite common.

thanks for the good report.  In order to be able to import the test.py
module py.test adds the parent directory to sys.path, thus the source
module.py is found when it is imported.  For running py.test (or
nosetests which does the same) standalone this sys.path hackery works.

Most people have their tests and package in subdirs.
If you put test.py into a subdir, does it work?

FWIW I once payed with tox accepting a pattern for test files
and copy them to a fresh directory and then run tests there.
This would avoid the issue you are seeing i think.

holger


> 
> Testing the version in the source tree would hide potential packaging errors.
> I would like to test the *installed* version. The directory with the installed
> version is last in sys.path! Am I assumed to hack sys.path?
> 
> Minimal example
> ###############
> 
> $ hg manifest
> .hgignore
> module.py
> py-1.4.3.zip
> pytest-2.0.3.zip
> setup.py
> test.py
> tox.ini
> 
> $ cat setup.py
> from distutils.core import setup
> setup(
>     name = "module",
>     version = "0.1",
>     py_modules = ["module"],
>     )
> 
> $ cat module.py
> member = 42
> 
> $ cat test.py
> import sys
> from os import path
> from pprint import pprint
> import module
> def test_member():
>     assert module.member == 42
> def test_path():
>     pprint(sys.path)
>     pprint(module)
>     expected =  ".tox/py26/lib/python2.6/site-packages/module.pyc"
>     assert path.exists(expected)
>     actual = path.relpath(module.__file__)
>     assert actual == expected
> 
> $ cat tox.ini
> [testenv]
> commands = py.test --verbose test.py
> deps =
>     {toxinidir}/py-1.4.3.zip
>     {toxinidir}/pytest-2.0.3.zip
> 
> $ tox --recreate -e py26
> _________________________________ [tox sdist] __________________________________
> [TOX] ***creating sdist package
> [TOX] /home/steinale/work/toxtests$ /usr/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
> [TOX] ***copying new sdistfile to '/home/steinale/.tox/distshare/module-0.1.zip'
> ______________________________ [tox testenv:py26] ______________________________
> [TOX] ***recreating virtualenv py26 (configchange/incomplete install detected)
> [TOX] /home/steinale/work/toxtests/.tox$ /usr/local/bin/virtualenv --distribute --no-site-packages -p /usr/bin/python2.6 py26 >py26/log/0.log
> [TOX] ***installing dependencies: /home/steinale/work/toxtests/py-1.4.3.zip, /home/steinale/work/toxtests/pytest-2.0.3.zip
> [TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download /home/steinale/work/toxtests/py-1.4.3.zip /home/steinale/work/toxtests/pytest-2.0.3.zip >1.log
> [TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download ../../dist/module-0.1.zip >2.log
> [TOX] /home/steinale/work/toxtests$ .tox/py26/bin/py.test --verbose test.py
> ============================= test session starts ==============================
> platform linux2 -- Python 2.6.5 -- pytest-2.0.3 -- /home/steinale/work/toxtests/.tox/py26/bin/python2.6
> collecting ... collected 2 items
> 
> test.py:5: test_member PASSED
> test.py:7: test_path FAILED
> 
> =================================== FAILURES ===================================
> __________________________________ test_path ___________________________________
> 
>     def test_path():
>         pprint(sys.path)
>         pprint(module)
>         expected =  ".tox/py26/lib/python2.6/site-packages/module.pyc"
>         assert path.exists(expected)
>         actual = path.relpath(module.__file__)
> >       assert actual == expected
> E       assert 'module.pyc' == '.tox/py26/lib/python2....te-packages/module.pyc'
> E         - module.pyc
> E         + .tox/py26/lib/python2.6/site-packages/module.pyc
> 
> test.py:13: AssertionError
> ------------------------------- Captured stdout --------------------------------
> ['/home/steinale/work/toxtests',
>  '/home/steinale/work/toxtests/.tox/py26/bin',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages/distribute-0.6.19-py2.6.egg',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages/pip-1.0.1-py2.6.egg',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/plat-linux2',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-tk',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-old',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-dynload',
>  '/usr/lib/python2.6',
>  '/usr/lib/python2.6/plat-linux2',
>  '/usr/lib/python2.6/lib-tk',
>  '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages']
> <module 'module' from '/home/steinale/work/toxtests/module.pyc'>
> ====================== 1 failed, 1 passed in 0.03 seconds ======================
> [TOX] ERROR: InvocationError: '.tox/py26/bin/py.test --verbose test.py'
> ________________________________ [tox summary] _________________________________
> [TOX] ERROR: py26: commands failed
> 
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>