[TIP] Calculating coverage of runtime-generated functions
ned at nedbatchelder.com
Sat May 2 17:12:15 PDT 2009
Nicolas Trangez wrote:
> Hey Ned, all,
> On 02 May 2009, at 23:24, Ned Batchelder wrote:
>> My initial impression is that you don't have two functions here, at
>> least not as far as the source is concerned. You're in the same
>> position as if your code were:
>> def fun(n, a):
>> return n * a
>> class DemoTest(unittest.TestCase):
>> def test_two(self):
>> self.assertEquals(fun(2, 2), 4)
>> There are cases in your code you haven't tested (the case where n is 3)
>> either way.
> Of course. The code I provided was just an example, where the amount
> of possible generated functions is endless, were the actual scenario
> has a finite number of generated functions.
Infinite values vs. finite values is a minor point: the problem is still
that you want more than statement coverage, you want value coverage
(measurement of which values have been covered). None of the Python
coverage tools provide this feature.
> Hopes all of this makes some sense...
Yes, it does, but it doesn't change my view of the problem. You have
code where every statement is executed, but you know that there are
interesting tests that haven't been executed. That's because you
understand that the important states of the program include not just the
program counter (the statements that have been executed), but also the
values of the variables.
It would be wonderful to have a tool that could help measure value
coverage, but Coverage is not it. If you have ideas about how to
automate value coverage measurement, I'd love to hear them. I've given
it no thought, but it could be very valuable.
Ned Batchelder, http://nedbatchelder.com
More information about the testing-in-python