[TIP] Getting combined coverage with Tox

Ned Batchelder ned at nedbatchelder.com
Fri Jun 5 06:30:31 PDT 2015


André:

You seem concerned about having to list all the environments in then 
envlist setting, but to my eye, the problem in your file is having the 
[testenv:py27] section as well as a [testenv:py34] section.  The look 
identical, unless I am missing something, so why not collapse them into 
one [testenv] section?  Then the envlist line is the only place you have 
to mention the python versions, and keeping that up-to-date should be no 
big deal.

--Ned.

On 6/5/15 12:15 AM, André Caron wrote:
> Hi all,
>
> I thought it would be somewhat easier to manage the list of 
> environment names if we could specify them on multiple lines, so I 
> sent in a quick pull request (#163) to add support for this.  Please 
> let me know what you think!
>
> Cheers,
>
> André
>
> On Thu, Jun 4, 2015 at 11:09 PM, André Caron <andre.l.caron at gmail.com 
> <mailto:andre.l.caron at gmail.com>> wrote:
>
>     Hi Randy,
>
>     Thanks for your quick response :-)
>
>     That repo you linked to is essentially using the same technique,
>     though it adds a neat way to clear the coverage before running the
>     tests.  As I state previously, this kind of degenerates when you
>     have multiple explicit environments as you need to list each of
>     them in the `envlist` setting.
>
>     Does anybody on this list have any plans for a built-in
>     pre-/post-command Tox setting or some kind of a plug-in interface
>     that could allow me to achieve the same effect without explicitly
>     listing all environments in the `envlist` setting?
>
>     Thanks,
>
>     André
>
>     On Thu, Jun 4, 2015 at 10:08 AM, Randy Syring <randy at thesyrings.us
>     <mailto:randy at thesyrings.us>> wrote:
>
>         I asked this on the list not too long ago and here is what I
>         found out:
>
>>             Hey Randy,
>>
>>             in the upcoming pytest-cov 2.0 you can easily create
>>             different data files for each tox environment and
>>             automatically combine them at the end of the tox run. See
>>             here for an example project:
>>             https://github.com/schlamar/pytest-cov/tree/2.0/example-tox-project
>>
>>             Unfortunately, there is no release date yet as I'm pretty
>>             busy right now. However, the 2.0 branch should be quite
>>             stable, so you could give it a try.
>>
>>             Marc
>>
>
>
>         *Randy Syring*
>         Husband | Father | Redeemed Sinner
>
>         /"For what does it profit a man to gain the whole world
>         and forfeit his soul?" (Mark 8:36 ESV)/
>
>         On 06/04/2015 12:02 AM, André Caron wrote:
>>         Hi all,
>>
>>         I've been searching for a recipe for computing combined code
>>         coverage while running Tox to against multiple versions of
>>         dependencies.
>>
>>         I found an old thread on this list (dates back to Aug 2011)
>>         that discusses this, but there is no clear conclusion:
>>
>>         http://lists.idyll.org/pipermail/testing-in-python/2011-August/004310.html
>>
>>         I've managed to hack something using some hints from there
>>         and some other tips here and there.  Here's a stripped-down
>>         version of what I'm using ATM:
>>
>>             [tox]
>>             envlist = py34,py27,coverage
>>
>>             [testenv:py34]
>>             deps =
>>               nose
>>               coverage
>>             setenv =
>>             COVERAGE_FILE=.coverage.{envname}
>>             commands =
>>               python setup.py check
>>               coverage run -m nose greet
>>
>>             [testenv:py27]
>>             deps =
>>               nose
>>               coverage
>>             setenv =
>>             COVERAGE_FILE=.coverage.{envname}
>>             commands =
>>               python setup.py check
>>               coverage run -m nose greet
>>
>>             [testenv:coverage]
>>             deps =
>>               coverage
>>             commands =
>>               coverage combine
>>               coverage report --show-missing --fail-under=100
>>
>>
>>         Notice that I'm using the "envlist" setting to force the
>>         "coverage" environment to run last.  This has the downside
>>         that I must list *all* environments there.  Appart from being
>>         tedious to read (CSV on one single line), I get the risk that
>>         someone forgets to add a new environment's name to the list.
>>
>>         Do you have any suggestions or know of any alternative ways
>>         of doing this?
>>
>>         Thanks!
>>
>>         André
>>
>>
>>         _______________________________________________
>>         testing-in-python mailing list
>>         testing-in-python at lists.idyll.org  <mailto:testing-in-python at lists.idyll.org>
>>         http://lists.idyll.org/listinfo/testing-in-python
>
>
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20150605/c6adddb3/attachment-0001.htm>


More information about the testing-in-python mailing list