[TIP] tox and coverage across multiple envs

[Ned Batchelder]

Andrew,

Coverage.py v3.5.1b1 is now available: 
http://pypi.python.org/pypi/coverage/3.5.1b1

The only feature change is a new config file section, [paths], that 
specifies how data from differently-named directories across machines 
should be combined.  The doc changes for this feature are at 
https://bitbucket.org/ned/coveragepy/changeset/259f543d8acd

I hope this will work to solve your problem.  Try it out when you get a 
chance, and let me know if something needs to be improved.  In 
particular, combining across OSes is a concern.

--Ned.

-------------------------------------------------------------------------------------------

On 8/10/2011 7:51 AM, Ned Batchelder wrote:
> On 8/10/2011 3:42 AM, Andrew Dalke wrote:
>> Hi Ned,
>>
>> On Aug 10, 2011, at 2:32 AM, Ned Batchelder wrote:
>>> I don't understand what those file names are that you're showing.
>>> You should have gotten data files like .coverage.machine.pid.12345 .
>>> And then "coverage combine" should have created a single .coverage
>>> file from them.  I'm not familiar with tox, is there something it
>>> does to affect the coverage collection?
>> Each tox environment has a name, like "py27-oe174" or "py27-ob23svn1",
>> and an associated configuration. Each configuration may have
>> dependencies. Tox makes a virtualenv for py27-oe174 under the
>> directory (this includes any extra download/installs)
>>
>>    ./tox/py27-oe174
>>
>> To run an environment, it appears that tox does a "setup.py install"
>> into the associated directory. For this environment the module
>> is put into
>>
>>    .tox/py27-oe174/lib/python2.7/site-packages/chemfp
>>
>> but for another environment it's put into
>>
>>    .tox/py27-ob23svn1/lib/python2.7/site-packages/chemfp/
>>
>>
>> Then because of the tox "commands" setting of
>>    coverage run -p ./tests/unit2 discover -s tests
>>
>> tox runs the environment specific "coverage".
>>
>> This puts the various .coverage.machine.pid.12345 files into
>> my top-level directory, which I can collect.
>>
>> The problem is that each tox environment has its own PYTHONPATH,
>> so that "from chemfp import openbabel" refers to
>>
>>    .tox/py27-oe174/lib/python2.7/site-packages/chemfp/openbabel.py
>> -or-
>>    .tox/py27-ob23svn1/lib/python2.7/site-packages/chemfp/openbabel.py
>> -or-
>>    ...
>>
>> depending on the environment that I run. This is how tox
>> maintains the needed isolation.
>>
>> The problem is that I want to merge those different reports
>> into one. In my case I can get what I want by removing
>>
>>    .tox/.*/site-packages/
>>
>> though I don't know if that's a generally useful practice.
>>
>> Perhaps there's an easy way to hack the .coverage.* format?
> Ah, now I see!  Yes, on my list for the near-term is to provide a way 
> to alias different trees so that you can tell coverage.py that all the 
> trees are equivalent, so that combine will work properly.  This is 
> issue #17: 
> https://bitbucket.org/ned/coveragepy/issue/17/support-combining-coverage-data-from 
> , although the patch on that ticket is not the way I would fix it.
>
> My plan is to provide a new option, similar to --source, which 
> specifies a point in the source tree.  This would get recorded in the 
> .coverage file.  Then combine would "line up" the specified points, 
> and all file paths would be considered relative to it.  Ideally you'd 
> be able to specify, ".tox/*/site-packages" in your case, and then 
> combine would work properly.
>
> --Ned.
>
>>                 Andrew
>>                 dalke at dalkescientific.com
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>