[TIP] tox and coverage across multiple envs

[Ned Batchelder]

On 8/10/2011 3:42 AM, Andrew Dalke wrote:
> Hi Ned,
>
> On Aug 10, 2011, at 2:32 AM, Ned Batchelder wrote:
>> I don't understand what those file names are that you're showing.
>> You should have gotten data files like .coverage.machine.pid.12345 .
>> And then "coverage combine" should have created a single .coverage
>> file from them.  I'm not familiar with tox, is there something it
>> does to affect the coverage collection?
> Each tox environment has a name, like "py27-oe174" or "py27-ob23svn1",
> and an associated configuration. Each configuration may have
> dependencies. Tox makes a virtualenv for py27-oe174 under the
> directory (this includes any extra download/installs)
>
>    ./tox/py27-oe174
>
> To run an environment, it appears that tox does a "setup.py install"
> into the associated directory. For this environment the module
> is put into
>
>    .tox/py27-oe174/lib/python2.7/site-packages/chemfp
>
> but for another environment it's put into
>
>    .tox/py27-ob23svn1/lib/python2.7/site-packages/chemfp/
>
>
> Then because of the tox "commands" setting of
>    coverage run -p ./tests/unit2 discover -s tests
>
> tox runs the environment specific "coverage".
>
> This puts the various .coverage.machine.pid.12345 files into
> my top-level directory, which I can collect.
>
> The problem is that each tox environment has its own PYTHONPATH,
> so that "from chemfp import openbabel" refers to
>
>    .tox/py27-oe174/lib/python2.7/site-packages/chemfp/openbabel.py
> -or-
>    .tox/py27-ob23svn1/lib/python2.7/site-packages/chemfp/openbabel.py
> -or-
>    ...
>
> depending on the environment that I run. This is how tox
> maintains the needed isolation.
>
> The problem is that I want to merge those different reports
> into one. In my case I can get what I want by removing
>
>    .tox/.*/site-packages/
>
> though I don't know if that's a generally useful practice.
>
> Perhaps there's an easy way to hack the .coverage.* format?
Ah, now I see!  Yes, on my list for the near-term is to provide a way to 
alias different trees so that you can tell coverage.py that all the 
trees are equivalent, so that combine will work properly.  This is issue 
#17: 
https://bitbucket.org/ned/coveragepy/issue/17/support-combining-coverage-data-from 
, although the patch on that ticket is not the way I would fix it.

My plan is to provide a new option, similar to --source, which specifies 
a point in the source tree.  This would get recorded in the .coverage 
file.  Then combine would "line up" the specified points, and all file 
paths would be considered relative to it.  Ideally you'd be able to 
specify, ".tox/*/site-packages" in your case, and then combine would 
work properly.

--Ned.

> 				Andrew
> 				dalke at dalkescientific.com
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>