[TIP] tox + py.test: test installed sdist
holger krekel
holger at merlinux.eu
Wed Jul 6 08:11:17 PDT 2011
Hi Stony,
On Wed, Jul 06, 2011 at 15:04 +0200, Alexander Steinert wrote:
> holger krekel, Mon 2011-07-04 22:10 CEST:
> > > there one thing in tox I don't quite understand yet. Even though tox
> > > creates the sdist and installs it into the virtual env, the
> > > installed version is not used/tested if tox is started in the
> > > project's root and the source of the module/package to be tested
> > > lives there too -- which should be quite common.
> >
> > thanks for the good report. In order to be able to import the test.py
> > module py.test adds the parent directory to sys.path, thus the source
> > module.py is found when it is imported. For running py.test (or
> > nosetests which does the same) standalone this sys.path hackery works.
> >
> > Most people have their tests and package in subdirs.
> > If you put test.py into a subdir, does it work?
>
> No. I changed my example to be less minimal and more realistic (see
> below).The result is the same. (Abs. path to) . is first in sys.path and
> ./.tox/py26/lib/python2.6/site-packages is last.
This results from your package/tests/__init__.py file which will
lead py.test to add the parent dir of package to sys.path
before importing it. See the "Note" in this section:
http://doc.pytest.org/en/latest/goodpractises.html?highlight=good%20practises#choosing-a-test-layout-import-rules
If you remove package/tests/__init__.py then your test will pass because
only 'package/tests' is added to sys.path in order to import test_module.
> IMHO, tox should take care that, during tests, modules are imported from
> the installed sdist. Imagine your setup.py is flaky, say you package
> no/wrong data files. You would never see that when you run tox.
I am not sure how tox could check that modules are imported from
the installed sdist. I agree, however, that it's too easy to
test against the inlined package instead of the installed one :/
> > FWIW I once payed with tox accepting a pattern for test files
> > and copy them to a fresh directory and then run tests there.
> > This would avoid the issue you are seeing i think.
>
> Hmm, even if I use changedir=package to let tox call py.test there,
> toxinidir is first in path :-(
Yes, that doesn't help. If we modified py.test to append rather
than prepend to sys.path we would run into issues as well because
"package.tests.test_module" would not be present in the installed
sdist.
best,
holger
> Regards
> Stony
>
>
> Excerpt from the modified example:
>
> $ hg manifest
> .hgignore
> package/__init__.py
> package/module.py
> package/tests/__init__.py
> package/tests/test_module.py
> py-1.4.3.zip
> pytest-2.0.3.zip
> setup.py
> tox.ini
>
> $ cat setup.py
> from distutils.core import setup
> setup(
> name = "package",
> version = "0.1",
> packages = ["package"],
> )
>
> $ cat tox.ini
> [testenv]
> commands = py.test --verbose package/tests
> deps =
> {toxinidir}/py-1.4.3.zip
> {toxinidir}/pytest-2.0.3.zip
>
> $ tox --recreate -e py26
> _________________________________ [tox sdist] __________________________________
> [TOX] ***creating sdist package
> [TOX] /home/steinale/work/toxtests$ /usr/bin/python setup.py sdist --formats=zip --dist-dir .tox/dist >.tox/log/0.log
> [TOX] ***copying new sdistfile to '/home/steinale/.tox/distshare/package-0.1.zip'
> ______________________________ [tox testenv:py26] ______________________________
> [TOX] ***recreating virtualenv py26 (configchange/incomplete install detected)
> [TOX] /home/steinale/work/toxtests/.tox$ /usr/local/bin/virtualenv --distribute --no-site-packages -p /usr/bin/python2.6 py26 >py26/log/0.log
> [TOX] ***installing dependencies: /home/steinale/work/toxtests/py-1.4.3.zip, /home/steinale/work/toxtests/pytest-2.0.3.zip
> [TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download /home/steinale/work/toxtests/py-1.4.3.zip /home/steinale/work/toxtests/pytest-2.0.3.zip >1.log
> [TOX] /home/steinale/work/toxtests/.tox/py26/log$ ../bin/pip install --download-cache=/home/steinale/work/toxtests/.tox/_download ../../dist/package-0.1.zip >2.log
> [TOX] /home/steinale/work/toxtests$ .tox/py26/bin/py.test --verbose package/tests
> ============================= test session starts ==============================
> platform linux2 -- Python 2.6.5 -- pytest-2.0.3 -- /home/steinale/work/toxtests/.tox/py26/bin/python2.6
> collecting ... collected 2 items
>
> package/tests/test_module.py:5: test_member PASSED
> package/tests/test_module.py:7: test_path FAILED
>
> =================================== FAILURES ===================================
> __________________________________ test_path ___________________________________
>
> def test_path():
> pprint(sys.path)
> pprint(module)
> expected = ".tox/py26/lib/python2.6/site-packages/package/module.pyc"
> assert path.exists(expected)
> actual = path.relpath(module.__file__)
> > assert actual == expected
> E assert 'package/module.pyc' == '.tox/py26/lib/pyth...package/module.pyc'
> E - package/module.pyc
> E + .tox/py26/lib/python2.6/site-packages/package/module.pyc
>
> package/tests/test_module.py:13: AssertionError
> ------------------------------- Captured stdout --------------------------------
> ['/home/steinale/work/toxtests',
> '/home/steinale/work/toxtests/.tox/py26/bin',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages/distribute-0.6.19-py2.6.egg',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages/pip-1.0.1-py2.6.egg',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/plat-linux2',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-tk',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-old',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/lib-dynload',
> '/usr/lib/python2.6',
> '/usr/lib/python2.6/plat-linux2',
> '/usr/lib/python2.6/lib-tk',
> '/home/steinale/work/toxtests/.tox/py26/lib/python2.6/site-packages']
> <module 'package.module' from '/home/steinale/work/toxtests/package/module.pyc'>
> ====================== 1 failed, 1 passed in 0.03 seconds ======================
> [TOX] ERROR: InvocationError: '.tox/py26/bin/py.test --verbose package/tests'
> ________________________________ [tox summary] _________________________________
> [TOX] ERROR: py26: commands failed
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
More information about the testing-in-python
mailing list