[TIP] condition coverage?

[Ned Batchelder]

Dan, the XML file uses the Cobertura file format, which used the word 
"condition-coverage".  The data in the XML report is the same as in the 
other reports, just presented differently.

Coverage.py can only measure lines, and branches between lines, because 
of the nature of the Python trace function.  It has no visibility into 
the behavior within a line.  So it can't tell which parts of "if a or b 
or c:" got run.

There is an old idea for a hackish way to get information like that, but 
there would still be significant work to make it usable: 
http://nedbatchelder.com/blog/200804/wicked_hack_python_bytecode_tracing.html

--Ned.


On 5/31/16 12:59 PM, Dan Stromberg wrote:
>
>
> On Tue, May 31, 2016 at 9:41 AM, Florian Bruhin <me at the-compiler.org 
> <mailto:me at the-compiler.org>> wrote:
>
>     * Dan Stromberg <dstromberglists at gmail.com
>     <mailto:dstromberglists at gmail.com>> [2016-05-31 09:17:33 -0700]:
>     > I'm looking for a way of getting a condition coverage report for python
>     > code.
>     >
>     > We've been using coverage.py .
>     >
>     > coverage.py appears to, at best, give branch coverage in its
>     HTML report.
>     > However, it's beginning to look like coverage.py can give condition
>     > coverage (which is what we prefer). but only via its XML report.
>     >
>     > Is that accurate?
>
>     I don't think so.
>
>     AFAIK there's a blogpost by Ned Batchelder about why he hasn't been
>     able to implement this, but I can't find it right now...
>
>
> I'm puzzled by this:
> $ egrep condition coverage.xml | head -5
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="24" 
> number="23"/>
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="29" 
> number="28"/>
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="99" 
> number="98"/>
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="36" 
> number="14"/>
>                                                 <line branch="true" 
> condition-coverage="100% (2/2)" hits="1" number="18"/>
>
> $ rm coverage.xml
>
> $ ./bin/jenkins-test > /dev/null 2>&1
>
> $ egrep condition coverage.xml | head -5
> cmd output started 2016 Tue May 31 09:52:23 AM PDT
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="24" 
> number="23"/>
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="29" 
> number="28"/>
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="99" 
> number="98"/>
>                                                 <line branch="true" 
> condition-coverage="50% (1/2)" hits="1" missing-branches="36" 
> number="14"/>
>                                                 <line branch="true" 
> condition-coverage="100% (2/2)" hits="1" number="18"/>
>
> ./bin/jenkins-test is using nose with --with-coverage - so via the 
> coverage.py plugin .
>
> Note the presence of the string "condition-coverage".
>
> Also, coverage.py's xmlreport.py has:
>                     xline.setAttribute(
>                         "condition-coverage",
>                         "%d%% (%d/%d)" % (100*taken/total, taken, total)
>                         )
>
>
> Thanks!
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20160531/20cd18f2/attachment.htm>