[TIP] Getting Weird Coverage Reports

[holger krekel]

On Fri, Nov 25, 2011 at 15:56 +0000, Michael Foord wrote:
> On 25/11/2011 12:22, Alfredo Deza wrote:
> >
> >
> >On Fri, Nov 25, 2011 at 7:07 AM, John Anderson <sontek at gmail.com
> ><mailto:sontek at gmail.com>> wrote:
> >
> >
> >
> >    On Fri, Nov 25, 2011 at 6:00 AM, Michael Foord
> >    <michael at voidspace.org.uk <mailto:michael at voidspace.org.uk>> wrote:
> >
> >        On 25/11/2011 11:55, John Anderson wrote:
> >>        I have a base class that all my models inherit from:
> >>
> >>        http://paste2.org/p/1796621
> >>
> >>        and majority of my tests are touching models and I even have
> >>        tests against this specific class:
> >>
> >>        http://paste2.org/p/1796622
> >>
> >>        Yet, when I run a coverage test in py.test:
> >>        py.test --cov-report=term-missing --cov app/models.py
> >>
> >>        app/models      46     23    50%   1-27, 37-40, 59, 65, 76, 82-84
> >>
> >>
> >>        How are lines 1-27 not covered?
> >
> >        All of those lines (imports, class definition, decorator
> >        application) are executed at *import time* (i.e. when the
> >        module is first created). The fact that those lines are
> >        reported as not covered implies to me that you are starting
> >        the test run (and therefore importing your models) before
> >        coverage is started.
> >
> >        You need to run your tests under the coverage tool, or ensure
> >        that coverage tracing is started before any imports execute.
> >
> >
> >    Ahh, So I have some py.test configuration, so that a test database
> >    is set to a clean slate when I start my tests, I thought
> >    pytest_configure would be late enough, but I guess its not?  This
> >    is what i'm doing:
> >
> >    from test_settings import settings
> >    from pyramid.config import Configurator
> >    from app.models import register_models
> >    from app.models import Entity
> >
> >
> >    def pytest_configure():
> >        url = settings.get('sqlalchemy.url')
> >        echo = settings.get('sqlalchemy.echo')
> >
> >        print 'Creating the tables on the test database %s' % url
> >
> >        config = Configurator(settings=settings)
> >
> >        register_models(config)
> >
> >        Entity.setup_database(url, echo=echo, drop=True, create=True)
> >
> >
> >Hi John,
> >
> >This is a similar problem that the Pecan [0] framework had
> >(still?) for exposing configuration setup for the actual
> >application
> >at test time.
> >
> >The problem is (as you suspect) that the configuration in
> >pytest_configure happens *before* any of the mechanisms for
> >coverage start.
> 
> It is worth filing an issue against py.test. It may be possible to
> fix in future versions so that coverage tracing starts before
> configuration is done.

Indeed!  Probably Meme could just enhance the coverage plugin by 
adding a "@pytest.mark.tryfirst" before its own pytest_configure() which
should be enough to get coverage started earlier.

best,
holger