[TIP] Getting combined coverage with Tox

André Caron andre.l.caron at gmail.com
Sat Jun 6 15:31:31 PDT 2015


Hi,

@Ned: My real application has several testenv sections with different
dependencies and different testing commands, so it's not as easy to fold
into a single generative test environment definitions.

@Carl: Nice to know that the multi-line support is already possible.  It
feels kind of weird to have commas on that multi-line list and not on other
multi-line lists though.

To sum it up, it seems like there are currently three techniques to compute
combined coverage:
1) add two coverage test environments and force them to run before and
after all other environments;
2) run coverage in append mode; and
3) use a meta-automation script (e.g. make).

I'll stick with #1 since it provides the least confusion and doesn't
require a meta-script.

Thanks for your help,

André

On Fri, Jun 5, 2015 at 1:53 PM, Carl Meyer <carl at oddbird.net> wrote:

> Hi André,
>
> On 06/04/2015 10:15 PM, André Caron wrote:
> > I thought it would be somewhat easier to manage the list of environment
> > names if we could specify them on multiple lines, so I sent in a quick
> > pull request (#163) to add support for this.  Please let me know what
> > you think!
>
> Multi-line envlist is already possible (because multi-line settings are
> built in to the ConfigParser format), you just aren't allowed to omit
> the commas. Here's a typical tox.ini for me which uses a multi-line
> envlist:
> https://github.com/carljm/django-transaction-hooks/blob/master/tox.ini
>
> Regarding the larger question about needing to explicitly specify
> envlist, your approach to tox.ini feels backwards from what I usually
> use. I see envlist as the required canonical listing of all envs, and
> `testenv:...` sections as only needed for special cases. Now that we
> have the new environment components support (which you can see used
> extensively in the above tox.ini), I rarely have specific `testenv:...`
> sections at all, often just the envlist and then the generic `testenv`
> section.
>
> And regarding measuring coverage with tox, I don't bother with separate
> coverage files or combining, I just use `coverage run -a ...` as the
> test command. The downside of this is that it doesn't automatically
> erase before a run or automatically report after a run. I usually handle
> this just by scripting tox instead of running it directly; either on CI
> or locally via a Makefile.
>
> HTH,
>
> Carl
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20150606/fc8365d6/attachment.htm>


More information about the testing-in-python mailing list