[TIP] [tox] Using/building in environments with no internet access

Barry Warsaw barry at python.org
Wed Nov 28 15:00:43 PST 2012

On Nov 28, 2012, at 10:09 PM, holger krekel wrote:

>thanks for your efforts! more below ...

I can't tell you how many people have sent me +1s for working on making tox
available in the Debian ecosystem.  It's for purely selfish reasons, I assure
you. :)

>If there is anything specific where i can help, let me know.  I can think of
>- change tests using $HOME (not many i think) to monkeypatch it to some
>  tempdir
>- allow tox to operate with an non-existing $HOME

Solving the $HOME requirement is a minor issue, but it's an annoyance that
would be nice to fix.  Anything you can do here to avoid having to point $HOME
at a tempdir during the build, and tox's own tests would be great.

>- introduce a "--ignoredeps" or better a TOX_IGNORE_ALL_DEPS env 
>  variable so that tox.ini files don't need to be hacked
>Do you think this would help or do you have other suggestions?

That would definitely help, but I'm not sure it quite goes all the way.  We do
want tox to ignore tox.ini dependencies (and either or both of the switch and
envar would be fine, with a mild preference on the envar if it has to be one
or the other).

However, we also need tox to use the system Python's packages in those cases.
Maybe you're thinking about that too.  For tox itself, we have to ignore the
py and py.test deps, but also have to avoid downloading the tox .zip file
which I think it does so that it can `import tox`.  More on that below.

So if --ignoredeps/$TOX_IGNORE_ALL_DEPS just lets tox trust Python's sys.path,
I think that would work.  But note the use of dist-packages below.

>The PYTHONPATH issue i don't understand. Isn't that an issue
>for other python packages as well?

It's not, although we currently have no packages with build-time tests
depending on tox, because until now, we didn't have tox available.  Probably,
most packages-with-build-time-tests use `python setup.py test` or nosetests,
and because these use the system Python's sys.path, they all succeed just
fine, assuming of course, we've done our job to ensure the proper build
dependencies are installed.

For some reason, this doesn't work with tox's tests.

Without adding '.' the `import tox` fails.  This probably makes the most sense
because tox itself isn't installed at the time its tests are run.  So this
allows us to import from the unpacked zip.  It's cheating a bit, but seems
okay, and would only be required for tox's build, since it's a bootstrapping
problem.  Other packages would just build-depend on tox and then they should
be able to run the equivalent of:

    for short_py_version in all_available_versions:
        # E.g. 2.7 -> py27
        run('tox -e py{}'.format(short_py_version.replace('.', '')))

Without adding /usr/lib/pythonX.Y/dist-packages, you get lots of failures like
this when running tox's own tests:

____________________________ test_separate_sdist ______________________________

cmd = <conftest.Cmd instance at 0x1e2a7a0>
initproj = <function initproj at 0x1e23758>

    def test_separate_sdist(cmd, initproj):
        distshare = cmd.tmpdir.join("distshare")
        initproj("pkg123-0.7", filedefs={
            'tox.ini': """
            """ % distshare
        result = cmd.run("tox", "--sdistonly")
>       assert not result.ret
E       assert not 1
E        +  where 1 = <conftest.RunResult instance at 0x1e49248>.ret

/build/tox-DkDcdQ/tox-1.4.2/tests/test_z_cmdline.py:417: AssertionError
------------------------------- Captured stdout --------------------------------
created project in /tmp/pytest-18/test_separate_sdist0/pkg123
/tmp/pytest-18/test_separate_sdist0/pkg123$ tox --sdistonly
------------------------------- Captured stderr --------------------------------
Traceback (most recent call last):
  File "/build/tox-DkDcdQ/tox-1.4.2/.tox/py27/bin/tox", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/build/tox-DkDcdQ/tox-1.4.2/.tox/py27/local/lib/python2.7/site-packages/distribute-0.6.30-py2.7.egg/pkg_resources.py", line 2807, in <module>
  File "/build/tox-DkDcdQ/tox-1.4.2/.tox/py27/local/lib/python2.7/site-packages/distribute-0.6.30-py2.7.egg/pkg_resources.py", line 690, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/build/tox-DkDcdQ/tox-1.4.2/.tox/py27/local/lib/python2.7/site-packages/distribute-0.6.30-py2.7.egg/pkg_resources.py", line 588, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: py>=1.4.9

'site-packages' here is a clue because on Debian, we install system packages
into dist-packages, and have our own hacked version of distribute to search
dist-packages in addition to site-packages (for both good and crazy reasons I
won't get into here ;).

So my guess is that maybe tox is installing its own version of distribute that
doesn't know about the Debian dist-packages thing?  Or, the tox in the
virtualenv's bin doesn't use the same sys.path as system Python.

>I could try to do a 1.4.3 release once we have figured out changes
>that would help.

One other thing that would be helpful actually, is if you could upload a
tar.gz to PyPI in addition to the .zip file.  We have to repack zips.  It's
not a showstopper, but a minor annoyance.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20121128/946f2ee4/attachment.pgp>

More information about the testing-in-python mailing list