[TIP] pytest: markers as functions

[holger krekel]

Hi Alexander,

On Mon, Jan 28, 2013 at 06:48 +0100, Alexander Steinert wrote:
> On Jan 26, 2013 3:07 PM, "holger krekel" <holger at merlinux.eu> wrote:
> > On Sat, Jan 26, 2013 at 14:08 +0100, Alexander Steinert wrote:
> > > @pytest.marker
> > > def verifies(request, requirement):
> > >     ....
> > >
> > > ....
> > >
> > > @verifies("Requirement 42")
> > > def test_42():
> > >     assert False
> > >
> > > ....
> > >
> > > $ py.test . --show-verifications
> > > test_42 vierifies Requirement 42
> > >
> > > By reading the fine manual I assume that this would be a feature
> request.
> > > What would come closest to it using the current pytest?
> >
> > I have an unfinished patch which allows roughly the above.  However, it's
> > not providing "request" and i am not sure how you would use it above.
> > Certainly at collection time it cannot be the "request" object that's used
> > during fixture setup.
> 
> I see. I need some object to collect which test verifies which
> requirement. That could be done without pytest help using a module level
> dict in my conftest.py. But I think that the marker decorated by
> pytest.marker (or pytest.markdefinition) would benefit from getting the
> marked test function (or something else) as its first argument so that
> it can get/derive the test name unique to the test session.

I see.  We could think about passing in "metafunc" which is also passed
to the pytest_generate_tests hook (called at collection time to implement
parametrization, also responsible for acting on the @pytest.mark.parametrize 
decorator).  It contains context (class/module/test function/...) information.
I am not sure if that's a good idea, though.  Let's see to play with it.
Can you create an enhancement issue so we can interact on this further?
Any help much appreciated, of course.

> Does pytest provide a hook just after collection took place? There I
> could evaluate my --show-verifications option, show my output and exit
> pytest.

Yes, you can use "pytest_collection_modifyitems" or maybe better
"pytest_collection_finish" to process all items and their markers
(to be found in "item.keywords['markername']").  Checkout _pytest/main.py's
"perform_collect" function and related ones if you need to know details.

> > In any case, here is a test taken from the unifinished patch which
> showcases
> > the intended API.
> [...]
> >             @pytest.mark.mymarker
> >             def test_1():
> >                 assert test_1.mymarker.getarg("arg1") == 42
> 
> What about test_1.mymarker.pargs and test_1.mymarker.kwargs, in case
> they should be forwarded somewhere?  Anyway, for my use case, I wouldn't
> need access to the marker from the test itself.

Sure, usually you access marker params from plugins/conftest files.
I showed an internal test which just verifies args are nicely accessible, not
meant as the way to use it.  As to forwarding, there shoudl still
be kwargs/args attributes on the marker to keep compatibility in any case.

best,
holger


> 
> /Stony