<div dir="ltr">tl;dr: Coverage with tox works best and usually as expected by setting usedevelop = True.<br><br>There is even a really simple way to combine various tox runs into one coverage result:<br> <br>- As said set `usedevelop = True`. Otherwise the coverage data will be collected in .tox/{envname}/site-packages and data can't be merged.<br>- Point COVERAGE_FILE to a specific file for each tox env (e.g. `.coverage.{envname}`)<br>- Have a special tox env which is run at the end to combine and report results, COVERAGE_FILE should point to `.coverage` here<br>- Optional, but recommended: run `coverage erase` in a first tox env (with `COVERAGE_FILE=.coverage`, too)<br> <br>This will be supported by pytest-cov in 2.0 (in development). But in general this should work with any test runner. pytest-cov doesn't work right now because it hard codes the data file so setting COVERAGE_FILE is useless.<br> <br>See a full example project at <a href="https://github.com/schlamar/pytest-cov/tree/2.0/example-tox-project">https://github.com/schlamar/pytest-cov/tree/2.0/example-tox-project</a>. <br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Nov 21, 2014 at 6:22 PM, Barry Warsaw <span dir="ltr"><<a href="mailto:barry@python.org" target="_blank">barry@python.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Nov 16, 2014, at 12:16 PM, Ned Batchelder wrote:<br>
<br>
>If I understand you correctly, this would require coverage to automatically<br>
>understand that .../site-packages/mailman is your own code. I don't see how<br>
>it could do that.<br>
<br>
Yes, it would have to know that `foo` and<br>
.tox/{envname}/lib/pythonX.Y/site-packages/foo were the same code<br>
<br>
It would be nice if 'pythonX.Y' were automatically detected, but I don't think<br>
that's a variable directly available from tox, so it would have to be<br>
calculated, probably from {envpython}.<br>
<br>
>> I also don't like having to explicitly omit everything else in the tox<br>
>> venv's site-packages. It seems like since coverage already omits stuff in<br>
>> the system's site-packages, it could do the same with the tox venv<br>
>> site-packages.<br>
><br>
>Usually the best way to omit things from site-packages is to use the<br>
>--source=. (or some other value) setting. In your case, it looks like it<br>
>would be:<br>
><br>
> [run]<br>
> source =<br>
> .<br>
> .tox/coverage/lib/python*/site-packages/mailman<br>
><br>
>Would that work?<br>
<br>
I'm not sure! I definitely get different coverage numbers, with more holes<br>
than I'd expect, but I am going to have to spend more time to investigate<br>
whether they are legitimate coverage misses or not. Still, thanks for the<br>
recommendation!<br>
<br>
Cheers,<br>
-Barry<br>
<br>_______________________________________________<br>
testing-in-python mailing list<br>
<a href="mailto:testing-in-python@lists.idyll.org">testing-in-python@lists.idyll.org</a><br>
<a href="http://lists.idyll.org/listinfo/testing-in-python" target="_blank">http://lists.idyll.org/listinfo/testing-in-python</a><br>
<br></blockquote></div><br></div>