[TIP] Adding a test made it quicker?

[Aaron Whitehouse]

Hello all,

Apologies, I seem to be half of the volume on this list at the moment!

We recently had an issue in our project where all of the tox tests
worked on our local machines, but failed on the Launchpad build server
(which runs very old versions of a number of packages).

I therefore would like to add a tox test environment that replicates the
Launchpad build server.

I updated the tox.ini to:

==
[tox]
envlist=py26,py27,lpbuildd

[testenv]
setenv=
    RUN_CODE_TESTS=1
deps=
    lockfile
    mock
    pexpect
    unittest2
# conditional deps (py26: unittest2) don't work on tox <1.8

[testenv:lpbuildd]
deps=
    lockfile==0.8
    mock==0.7.2
    pexpect==2.4
    python==2.7

commands=
    {envpython} {toxinidir}/setup.py test {posargs}

==

And this actually *appears* to run as expected, in that it says the
commands for each run succeeded, but running the three test environments
is only taking half as long as the two environments used to take, so I'm
assuming something isn't working correctly. It also only lists each test
name once in the output, rather than once per environment (as it used
to). Any ideas what could be going on?

Prior to changing the tox.ini, the time taken to run the tests was as
follows:

----------------------------------------------------------------------
Ran 382 tests in 333.654s

OK
___________________________________ summary
____________________________________
  py26: commands succeeded
  py27: commands succeeded
  congratulations :)

real    11m4.173s
user    5m3.300s
sys        1m49.576s


After I changed the tox.ini, it was as follows:

----------------------------------------------------------------------
Ran 382 tests in 342.085s

OK
___________________________________ summary
____________________________________
  py26: commands succeeded
  py27: commands succeeded
  lpbuildd: commands succeeded
  congratulations :)

real    5m52.340s
user    2m44.028s
sys        0m59.100s

Kind regards,

Aaron