[TIP] pytest test collection problem with markers and test parametrization?

[Pella,Chris]

That may be true... but the only reason I was looking into this is because the test was actually being executed in our nightly Jenkins automation run. I was only using the -collectonly switch in to avoid running the tests, which take some time to run. The existing issue remains unresolved.

Chris

From: Jason Coutu [mailto:jcoutu at vendasta.com]
Sent: April-08-13 2:53 PM
To: Pella,Chris
Cc: testing-in-python at lists.idyll.org
Subject: Re: [TIP] pytest test collection problem with markers and test parametrization?

Collect all doesn't always report what you expect.  When you mark like that, it won't get run, but it still get's collected. When you run it will be "Skipped"  run with -rs

Jason

On Mon, Apr 8, 2013 at 12:38 PM, Pella,Chris <Chris.Pella at safenet-inc.com<mailto:Chris.Pella at safenet-inc.com>> wrote:
I asked this question in stackoverflow but didn't get any answers, so I'll narrowcast it here.
We have a marked test that we expect to be not executed because py.test was invoked with another marker, yet the test is getting executed.
e.g.
@pytest.mark.stress
def test_one(some_fixture):
     pass

@pytest.mark.sa<http://mark.sa>
def test_two(some_fixture):
     pass

If we run with pytest -collectonly -m "sa" it will show that both tests are collected, but we expected test_one to not be collected for execution. The only way to avoid the stress test is by invoking pytest --collectonly -m "sa and (not stress)".
We inspect the original argument list in the pytest_generate_tests hook to grab the markers so we can use the platform (e.g. sa in this case) marker in the fixture to set up the test resources, but otherwise don't manipulate the arguments.  Is the test parametrization hook below causing pytest to think test_one is marked with the 'sa' marker?

def pytest_generate_tests(metafunc):
    """
    Generate the parameterization list for the source device
    based on the markers set in the command line.
    @param metafunc - meta object for a specific function.
    """
    source    = [ 'unsupported' ]

    if '-m' in metafunc.config._origargs:
        index = metafunc.config._origargs.index('-m')
        markers = metafunc.config._origargs[index + 1]
        markers = markers.split(',')

        source = []
        if 'sa' in markers:
            source.append( 'sa' )
        if 'g5' in markers:
            source.append( 'g5' )
        if 'pci' in markers:
            source.append( 'pci' )
        if len(source) == 0:
            source = [ 'unsupported' ]

    if 'source' in metafunc.funcargnames:
        metafunc.parametrize("source", source, indirect=True)
    if 'rped_source_only' in metafunc.funcargnames:
        metafunc.parametrize("rped_source_only", source, indirect=True)




Chris


The information contained in this electronic mail transmission

may be privileged and confidential, and therefore, protected

from disclosure. If you have received this communication in

error, please notify us immediately by replying to this

message and deleting it from your computer without copying

or disclosing it.





_______________________________________________
testing-in-python mailing list
testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>
http://lists.idyll.org/listinfo/testing-in-python


The information contained in this electronic mail transmission 
may be privileged and confidential, and therefore, protected 
from disclosure. If you have received this communication in 
error, please notify us immediately by replying to this 
message and deleting it from your computer without copying 
or disclosing it.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20130408/c629e50f/attachment-0001.htm>