[TIP] Coverage.py 4.1b2: re-written branch measurement

[Ned Batchelder]

On 1/25/16 12:01 PM, Chris Jerdonek wrote:
> On Mon, Jan 25, 2016 at 3:21 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
>> On 1/23/16 8:08 PM, Ned Batchelder wrote:
>>> For example:
>>>
>>>      try:                    # 1
>>>          ...                 # 2
>>>      except ValueError:      # 3
>>>          ...                 # 4
>>>      finally:                # 5
>>>          ...                 # 6
>>>
>>>      try:                    # 8
>>>          ...                 # 9
>>>      except:                 # 10
>>>          ...                 # 11
>>>      finally:                # 12
>>>          ...                 # 13
>>>
>>> In the first block of code, there's a branch from line 3 to line 6, in the
>>> case of an exception other than ValueError.  Should coverage.py flag that as
>>> missing if such an exception never happens?  Right now it does.  In the
>>> second block of code, coverage.py understands that line 10 can never jump to
>>> 13, because there is no exception that would skip line 11.
>>>
>> I forgot to mention: Coverage.py gives you a missed branch for exceptions
>> other than ValueError on line 3 above, but it won't if you don't have a
>> finally clause and the exception leaves the function:
>>
>>      def func():                 # 1
>>          try:                    # 2
>>              ...                 # 3
>>          except ValueError:      # 4
>>              ...                 # 5
>>
>> On the face of it, that seems wrong: why should the presence of a finally
>> clause change whether the except is considered to be partially covered or
>> not?
> Perhaps the answer could be: because coverage.py only considers
> branches _within_ functions (i.e. from one line to another).  In
> particular, it doesn't consider as branches the possibility of an
> arbitrary line of code raising an exception and leaving the function
> without being handled.
You are right that coverage.py won't consider every statement as a 
possible exception raiser.  But it does understand that "except" is 
executed when an exception happens.  And it could understand that there 
are possible exceptions that are not being handled, and therefore, a 
branch from the "except" to the function exit is possible, just as it 
understands that a branch from "except" to "finally" is possible.

But will people think this is overkill?

There are three possibilities, all having to do with branches from an 
"except" for the exceptions it doesn't handle:

1) None of them are considered for branch coverage.  This could mean 
that interesting code paths are overlooked by coverage.py
2) Only the branch from "except" to "finally" is considered, but not the 
branch from "except" to leaving the function.  This is the way 
coverage.py behaves in 4.1b2 today.
3) All branches from an "except" for unhandled exceptions are 
considered.  This could mean that branches are flagged as missing, when 
actually the developer knows they can't happen, and then has to sprinkle 
pragmas onto "except" to quiet them.

To me, the only reason to choose #2 is that the code already works that 
way.  It doesn't seem like a defensible position.  If you were choosing 
between #1 and #3, which would you choose?

--Ned.
>
> For example, in the following lines, an exception being raised at #1
> isn't (I think) considered a distinct branch.  (There is no
> surrounding exception handling in this example.):
>
>      ...  #1
>      ...  #2
>
> One reason this makes sense is that maybe in general it's not possible
> for an arbitrary line of code to raise an exception, and so it would
> be wrong to consider that branch missing.
>
> Come to think of it, that could be a better answer to the question
> above.  If you have a finally clause, it should be only because it is
> needed.  If an exception other than ValueError can't occur, then the
> finally clause can (and should?) be removed because it's not needed.
>
> --Chris
>
>
>
>> --Ned.
>>
>>> Try it, let me know what you think:
>>> https://pypi.python.org/pypi/coverage/4.1b2
>>>
>>> --Ned.
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python