[TIP] Getting Weird Coverage Reports

[meme dough]

Hi,

I will consider the implications of moving coverage start earlier, it
could be problematic.

However, in the first instance pytest_configure is very early and
there are 2 approaches you could take immediately.

First use pytest_sessionstart instead of pytest_configure for your
setup doing a multicall so that cov is called before your setup.
Without checking something along the lines of:

    def pytest_sessionstart(self,__multicall__, session):
        __multicall__.execute()
        # your setup here now that coverage is active

Second, use pytest funcargs for test fixtures.  So have a funcarg
which only once per session (cached_setup with session scope) does the
setup and teardown that you want.  IMHO funcargs are much better than
xunit style test fixtures.

Finally, I don't see any session scope xunit style test fixture.
Would it be worth considering adding one?  Some may point out that
hooks like pytest_configure or pytest_sessionstart could be used for
this purpose, but I think the purpose is actually different.  Having
session scope test fixture (funcarg or xunit style) gives a clear
place for test fixture code.

:)

On 26 November 2011 07:34, holger krekel <holger at merlinux.eu> wrote:
> On Fri, Nov 25, 2011 at 15:56 +0000, Michael Foord wrote:
>> On 25/11/2011 12:22, Alfredo Deza wrote:
>> >
>> >
>> >On Fri, Nov 25, 2011 at 7:07 AM, John Anderson <sontek at gmail.com
>> ><mailto:sontek at gmail.com>> wrote:
>> >
>> >
>> >
>> >    On Fri, Nov 25, 2011 at 6:00 AM, Michael Foord
>> >    <michael at voidspace.org.uk <mailto:michael at voidspace.org.uk>> wrote:
>> >
>> >        On 25/11/2011 11:55, John Anderson wrote:
>> >>        I have a base class that all my models inherit from:
>> >>
>> >>        http://paste2.org/p/1796621
>> >>
>> >>        and majority of my tests are touching models and I even have
>> >>        tests against this specific class:
>> >>
>> >>        http://paste2.org/p/1796622
>> >>
>> >>        Yet, when I run a coverage test in py.test:
>> >>        py.test --cov-report=term-missing --cov app/models.py
>> >>
>> >>        app/models      46     23    50%   1-27, 37-40, 59, 65, 76, 82-84
>> >>
>> >>
>> >>        How are lines 1-27 not covered?
>> >
>> >        All of those lines (imports, class definition, decorator
>> >        application) are executed at *import time* (i.e. when the
>> >        module is first created). The fact that those lines are
>> >        reported as not covered implies to me that you are starting
>> >        the test run (and therefore importing your models) before
>> >        coverage is started.
>> >
>> >        You need to run your tests under the coverage tool, or ensure
>> >        that coverage tracing is started before any imports execute.
>> >
>> >
>> >    Ahh, So I have some py.test configuration, so that a test database
>> >    is set to a clean slate when I start my tests, I thought
>> >    pytest_configure would be late enough, but I guess its not?  This
>> >    is what i'm doing:
>> >
>> >    from test_settings import settings
>> >    from pyramid.config import Configurator
>> >    from app.models import register_models
>> >    from app.models import Entity
>> >
>> >
>> >    def pytest_configure():
>> >        url = settings.get('sqlalchemy.url')
>> >        echo = settings.get('sqlalchemy.echo')
>> >
>> >        print 'Creating the tables on the test database %s' % url
>> >
>> >        config = Configurator(settings=settings)
>> >
>> >        register_models(config)
>> >
>> >        Entity.setup_database(url, echo=echo, drop=True, create=True)
>> >
>> >
>> >Hi John,
>> >
>> >This is a similar problem that the Pecan [0] framework had
>> >(still?) for exposing configuration setup for the actual
>> >application
>> >at test time.
>> >
>> >The problem is (as you suspect) that the configuration in
>> >pytest_configure happens *before* any of the mechanisms for
>> >coverage start.
>>
>> It is worth filing an issue against py.test. It may be possible to
>> fix in future versions so that coverage tracing starts before
>> configuration is done.
>
> Indeed!  Probably Meme could just enhance the coverage plugin by
> adding a "@pytest.mark.tryfirst" before its own pytest_configure() which
> should be enough to get coverage started earlier.
>
> best,
> holger
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>