[TIP] coverage combine with the results of a jenkins matrix job...

[Ned Batchelder]

Chris,

If you use the --parallel flag, the .coverage files will be given 
distinct names.  You can then copy them to a single directory, and 
"coverage combine" will combine them.  You'll need a [path] section if 
the same source file is found at different paths for different runs.

--Ned.

On 2/14/2013 6:25 PM, Chris Withers wrote:
> (copying the list, since I think this may be of wider interest)
>
> Hi Ned,
>
> Okay, I think I'm part of the way there, but still confused, hopefully 
> you can help...
>
> So, the runs generating the .coverage files are in this matrix:
>
> http://jenkins.simplistix.co.uk/view/testfixtures/job/testfixtures-buildout/ 
>
>
> Here's an example output:
>
> http://jenkins.simplistix.co.uk/view/testfixtures/job/testfixtures-buildout/PYTHON=3.3,label=linux/182/console 
>
>
> The actual command being run is:
>
> bin/nosetests --with-xunit --with-cov --cov=testfixtures 
> --cov-report=xml --cov-report=html
>
> So, now I have a downstream job where I want to combine all those 
> .coverage files and generate html and xml reports. Here's the 
> workspace for that job:
>
> http://jenkins.simplistix.co.uk/view/testfixtures/job/testfixtures-coverage/ws/ 
>
>
> So, each PYTHON* has a .coverage file in it, testfixtures-3.0.0dev 
> contains the source.
>
> I thought it would just be a case of:
>
> coverage combine PYTHON*/.coverage
> coverage xml
> coverage html
>
> But combine doesn't appear to work as I expect. In fact, I don't think 
> I have a clue how it's supposed to work, so thought I'd ask ;-)
> What should I be looking to do? I have a feeling I'll need a [path] 
> section in it, but no idea what I should put there...
>
> Any help very gratefully received!
>
> Chris
>
> On 13/02/2013 16:21, Ned Batchelder wrote:
>> Sorry, I'm not well-versed enough with Jenkins to know.
>>
>> --Ned.
>>
>> On 2/13/2013 10:57 AM, Chris Withers wrote:
>>> Hmm, any idea how to do that in a Jenkins matrix job?
>>>
>>> Chris
>>>
>>> On 13/02/2013 15:52, Ned Batchelder wrote:
>>>> I would run coverage twice: for Python 2, and again for Python 3, and
>>>> then I would combine the results (coverage combine).
>>>>
>>>> --Ned.
>>>>
>>>> On 2/13/2013 10:36 AM, Chris Withers wrote:
>>>>> One question: if I have code paths that should not be covered under a
>>>>> particular version of python, but should under others, how can I go
>>>>> about dealing with that?
>>>>>
>>>>> (eg:
>>>>>
>>>>> if PY3:
>>>>> ...
>>>>> else:
>>>>> ...
>>>>>
>>>>> both of those branches should be covered in different test runs, but
>>>>> neither of them will be covered in a single run.
>>>>>
>>>>> How would you tackle this to give meaningful output?
>>>>>
>>>>> cheers,
>>>>>
>>>>> Chris
>