[TIP] Calculating coverage of runtime-generated functions

Ned Batchelder ned at nedbatchelder.com
Sat May 2 14:24:35 PDT 2009

My initial impression is that you don't have two functions here, at 
least not as far as the source is concerned.  You're in the same 
position as if your code were:

def fun(n, a):
    return n * a

class DemoTest(unittest.TestCase):
    def test_two(self):
        self.assertEquals(fun(2, 2), 4)

There are cases in your code you haven't tested (the case where n is 3) 
either way.

My point is that the source lines are being covered in both your code 
and mine, but there are cases not covered because the actual values used 
in the functions matter.  I don't know how to deal with it though: how 
should coverage decide what values are sufficient to cover the cases?


Nicolas Trangez wrote:
> Hija,
> In a project I'm working on there are several constructs like the one  
> pasted at the end of this email. Recently we wanted to enhance the  
> test coverage of the project, and measure this as well, using a  
> standard code coverage tool.
> I've been looking at both coverage.py (3.0 beta) and figleaf, both  
> integrated in nose, which is the test runner we're using, but none of  
> them cover the problem, not to my surprise since it seems rather hard  
> to tackle. Given the code snippet below (which is obviously an  
> oversimplification of the real code), coverage tools report 100%  
> coverage, whilst function 'three' is not tested at all. If the  
> generated functions would use more complex components in their  
> closure, one really wants to test all of them.
> The main question is: is there any existing tool which allows to check  
> code coverage of code for which no 1-to-1 mapping to sourcecode is  
> available?
> If not I guess it'd be useful to work on this, but both from a  
> technical as well as from a UI point of view I expect some challenges.  
> Any ideas are welcome :-)
> Thanks,
> Nicolas
> def gen(n):
>      def fun(a):
>          return n * a
>      return fun
> two = gen(2)
> three = gen(3)
> import unittest
> class DemoTest(unittest.TestCase):
>      def test_two(self):
>          self.assertEquals(two(2), 4)
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python

Ned Batchelder, http://nedbatchelder.com

More information about the testing-in-python mailing list