[TIP] branch coverage

[m h]

On Jan 24, 2008 1:16 PM, Titus Brown <titus at caltech.edu> wrote:
> [ munch ]
>
> Andrew, this looks great!

Yeah, this is a great start.

>
> -> I could generate raw Python code, which would be ugly, have the
> -> comments stripped out, and line numbers changed.  Or I could generate
> -> byte code.
>
> I think byte code is cleaner.

I think generate both if you could.  The python code may be ugly, but
it does an excellent job of explaining how complicated code can
actually get.  I also serves well to illustrate the differences
between the different types of coverage (line, branch, path).

>
> -> If the latter, I was thinking to write a .py -> .pyc compiler, but do
> -> I use it like compileall?  Or do I generate the .pyc files in another
> -> directory, which is used for the coverage testing.  Where do I keep
> -> the coverage results?  Probably all in a single directly, named after
> -> the Python module name.
>
> Can you just generate .pyc files in the normal place, i.e. same
> directory?  How would this affect 'execfile'd code -- presumably you'd
> need to treat scripts differently...
>
> And, for the love of god, put a simple API on top of the coverage
> results.  Half the reason I wrote figleaf was that I needed to
> manipulate the output of coverage, and it was quite difficult to do so
> for several reasons.  Not having a query-able internal data structure is
> one of those reasons.

This is probably obvious, but the API needs to expose mapping from
branchs/paths covered back to original code.  (I was going to say back
to line number, but that's not sufficient.  Perhaps line number +
start/end column?)

>
> -> Do people only care about if the branch was true/false or are the
> -> number of tests also important?  What about the number of times a
> -> line was executed, vs. a flag saying that it was covered?
>
> IMO only, but I think people will want as much detail as possible.

Yes, for path coverage, I don't only want to know if the section of
code was executed, but I want to every section of code that was
executed per time a module was called.

Looking forward to seeing something I can play with ;)