[TIP] Conditional code coverage

[André Caron]

Hi all,

I finally got back to trying this and I hit a road block.  In order to use
this from Tox, I would need to do something like this:

[testenv]
setenv =
  COVERAGE_PLATFORM=...  # current platform here
commands =
  coverage erase
  coverage run -m ...
  coverage report ...


There's a V2 configuration specification[1] that would allow this:

[testenv]
setenv =
  win: COVERAGE_PLATFORM=win32
  linux: COVERAGE_PLATFORM=posix
  darwin: COVERAGE_PLATFORM=posix
commands =
  coverage erase
  coverage run -m ...
  coverage report ...


[1]: https://tox.readthedocs.io/en/latest/config-v2.html

However, that doesn't seem to be supported by Tox 2.3.1.

Any plans to implement this part of the Tox V2 configuration specification
soon?  If not, can I help out?

André

On Tue, Aug 2, 2016 at 11:21 AM, André Caron <andre.l.caron at gmail.com>
wrote:

> Yes, I know.  I've used the coverage combine function before, that's the
> easy part.  It's just that adding this to our CI infrastructure's automatic
> build status (which is my end goal) with this technique involves touching a
> BitBucket server plug-in, our legacy Anthill-based build machines and I
> don't know what else and this will take some time from another team that's
> really busy ATM.  We'll get there eventually, I'm just not that team's top
> priority right now.
>
> Honestly, finding a way to get 100% by ignoring platform-specific code is
> way easier for me in the short term.
>
> Thanks for the suggestion though :-)
>
> André
>
> On Tue, Aug 2, 2016 at 11:07 AM, Chris Withers <chris at withers.org> wrote:
>
>> If you can copy files from it, you can use coverage combine ;-)
>>
>> On 02/08/2016 15:23, André Caron wrote:
>>
>> @Chris: yeah, like I said in my original post, that's the ideal
>> approach.  I've worked with this technique using coveralls.io before and
>> I really like it.  Problem is I'm currently running on some corporate build
>> infrastructure (not my call) that's really slow to change and I won't be
>> able to get that quickly enough.  Was looking for some other ideas as a
>> temporary work around to buy some time.
>>
>> André
>>
>> On Tue, Aug 2, 2016 at 7:35 AM, Ned Batchelder <ned at nedbatchelder.com>
>> wrote:
>>
>>> This is definitely the most common approach.  Some people want to see
>>> that they have complete coverage on each platform independently, though I
>>> have never tried that myself.  André indicated he could not use combine for
>>> whatever reason, not sure why.
>>>
>>> --Ned.
>>>
>>> On 8/2/16 6:27 AM, Chris Withers wrote:
>>>
>>> When I hit this, I just ran the tests on the various platforms and used
>>> "coverage conbine" to get them into one place before making assertions.
>>>
>>> cheers,
>>>
>>> Chris
>>>
>>> On 27/07/2016 18:56, André Caron wrote:
>>>
>>> Neat!  Looks like a winner, at least until I have more than 2 platforms
>>> (this mechanism being based on exclusion, I have to "negate" the platform,
>>> which may yield multiple results).
>>>
>>> André
>>>
>>> On Wed, Jul 27, 2016 at 9:54 AM, Ned Batchelder <ned at nedbatchelder.com>
>>> wrote:
>>>
>>>> You can use environment variables in the .coveragerc file, so you can
>>>> create a pattern for pragmas that uses environmental data.  For example:
>>>> https://github.com/habnabit/ebb-lint/blob/master/.coveragerc :
>>>>
>>>> [report]
>>>> exclude_lines =
>>>>     pragma: no ${TOX_ENVNAME}
>>>>     pragma: no ?cover
>>>>
>>>> --Ned.
>>>>
>>>>
>>>> On 7/26/16 6:33 PM, André Caron wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I have some code base that has platform-specific code, mostly related
>>>> to how to handle SIGINT and CTRL-C for which I like to use `coverage report
>>>> --fail-under=100`.  Problem is that some code only runs on one OS, so
>>>> coverage is always partial in those cases and it's not so straightforward.
>>>>
>>>> I've used Coveralls for some open source projects and their solution
>>>> AFAICT seems to be to collect the `.coverage` files in a central location
>>>> and then combine the coverage files, after which you can run the actual
>>>> reporting.
>>>>
>>>> However, I'm not in a position to use Coveralls for this project and
>>>> integrating a distributed coverage combine is not viable in the
>>>> short/medium term.
>>>>
>>>> I'm looking for suggestions on a quick and dirty way to solve this.
>>>>
>>>> One thing I thought of was using coverage's `exclude_lines` option [1]
>>>> with a platform specific token to get this:
>>>>
>>>> if sys.platform == 'win32':  # pragma: cover win32
>>>>
>>>>
>>>> I guess this works, but it requires that I have a platform-specific
>>>> configuration file (e.g. via duplicatation or generation from a template).
>>>>
>>>> Anybody have any better ideas?
>>>>
>>>> Thanks,
>>>>
>>>> André
>>>>
>>>> [1]: https://coverage.readthedocs.io/en/coverage-4.
>>>> 1/excluding.html#advanced-exclusion
>>>>
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>>
>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20160814/e17ae423/attachment.htm>