[TIP] Getting Weird Coverage Reports

[holger krekel]

On Sun, Nov 27, 2011 at 01:34 +1100, meme dough wrote:
> yes, funcarg test fixtures support session scope which was the second
> suggestion.
> 
> but, xunit style test fixtures don't support session scope.  Should it
> be added with something along the lines of setup_session,
> teardown_session in addition to setup_module, teardown_module,
> setup_class, teardown_class etc
> 
> That keeps test fixture code in test fixture functions (either funcarg
> style or xunit style).
> 
> Hooks are kept for extending or altering pytest by plugins
> (pytest-xdist, pytest-cov) or projects (conftest).
> 
> All test fixture functions (either funcarg or xunit style) are invoked
> after pytest_sessionstart and before pytest_sessionfinish.

Right. I am thinking of calling a setup_directory|teardown_directory(X)
in respective conftest.py files.

FWIW there were some users on the IRC channels requesting the equivalent
of nose's setup_package/teardown_package.  I'd rather like to use a
different approach - call setup_directory(testdata) where "testdata"
is a global object on which you can set attributes. This very object
can be accessed from test functions as a funcarg:

    def test_function(testdata):
        # access globally shared test resources as attributes on "testdata"

it probably would also make sense to make the config object available
to setup_directory so that one can create global resources depending
on command line options etc. Either as an attribute on testdata or
as a second argument to the setup/teardown functions.

Makes sense to you?

best,
holger


> On 26 November 2011 23:34, holger krekel <holger at merlinux.eu> wrote:
> > On Sat, Nov 26, 2011 at 21:33 +1100, meme dough wrote:
> >> I will consider the implications of moving coverage start earlier, it
> >> could be problematic.
> >>
> >> However, in the first instance pytest_configure is very early and
> >> there are 2 approaches you could take immediately.
> >>
> >> First use pytest_sessionstart instead of pytest_configure for your
> >> setup doing a multicall so that cov is called before your setup.
> >> Without checking something along the lines of:
> >>
> >>     def pytest_sessionstart(self,__multicall__, session):
> >>         __multicall__.execute()
> >>         # your setup here now that coverage is active
> >
> > In fact, using "__multicall__" is still somewhat of an internal API and
> > nowadays it's better to write:
> >
> >    @pytest.mark.trylast
> >    def pytest_sessionstart(self, session):
> >
> > which should take care to execute this function as late as possible in
> > the chain of hook implementations.
> >
> >> Second, use pytest funcargs for test fixtures.  So have a funcarg
> >> which only once per session (cached_setup with session scope) does the
> >> setup and teardown that you want.  IMHO funcargs are much better than
> >> xunit style test fixtures.
> >
> > agreed.
> >
> >> Finally, I don't see any session scope xunit style test fixture.
> >> Would it be worth considering adding one?  Some may point out that
> >> hooks like pytest_configure or pytest_sessionstart could be used for
> >> this purpose, but I think the purpose is actually different.  Having
> >> session scope test fixture (funcarg or xunit style) gives a clear
> >> place for test fixture code.
> >
> > not sure what you mean -- funcargs allow a per-session scope, see
> > http://pytest.org/latest/funcargs.html?highlight=cached_setup#_pytest.python.FuncargRequest.cached_setup
> >
> > cheers,
> > holger
> >
>