[TIP] running tests from pytest.main

[Pella,Chris]

Delayed response... just got back from camping up the Ottawa river. Inline...

-----Original Message-----
From: holger krekel [mailto:holger at merlinux.eu] 
Sent: Tuesday, June 26, 2012 3:49 AM
To: Pella,Chris
Cc: testing-in-python at lists.idyll.org
Subject: Re: [TIP] running tests from pytest.main

On Mon, Jun 25, 2012 at 14:52 -0400, Pella,Chris wrote:
> We are invoking pytest using pytest.main from a custom functional test framework we are layering on top of pytest. It creates a global test configuration dictionary based on some command line switches and some config file data.  I was trying to prevent the global config dictionary from being re-initialized and making it a singleton by doing this:
> if "config" not in dir(sys.modules[__name__]):
>                 config = {}
>                 ....

Why are you going through the sys.modules indirection instead of just doing:

    # contained in some module
    config = {}

? 
[Pella,Chris] I found it wasn't working in my application the way I would have thought. We are setting some values in the config dictionary from the command line because they are constantly changing, and some settings which are usually fixed on each machine are read from a file. I was just using the sys.modules as a check to keep python from reading the config file every time a test file imported the config module. Also, I wanted to figure out what why the config variables that were set from the command line were gone when pytest took over, and I thought this would show me if pytest was changing sys.modules, resulting in what I was seeing. 
In the main framework module, we are setting some commandline options... I am just showing --product as an example.
<in the main module>
import config
parser = OptionParser()
    
parser.add_option("--product",
                      action = "store",
                      type = "string",
                      dest = "product",
                      help = "select product under test")
    
config['product'] = options.product

<config module>
# The config file has sections keyed by the 'product' option passed in the command line
if 'config' not in dir(sys.modules[__name__]):
    config = {}
    #Command line settings set to reasonable defaults
    #Config File settings
    config_parser = ConfigParser()
    config_parser.read("LunaConfig.ini")
    config.update(dict(config_parser._sections))
    
<main test runner module>
# Here is where we call pytest.main
pytest_command = pytest_options
if config.has_key('test_groups'):  
	pytest_command += "-v -m %s" %"".join(map(str,config.get('test_groups')))
if config.has_key('file_to_test'):
      pytest_command += " %s" %config.get('file_to_test')
pytest.main(pytest_command)

If this is done in a module, "config" automatically is a singleton because
all imported Python modules are singletons (i.e. a given module is only
imported once usually).

[Pella,Chris] I stepped through the code using the pdb and found that what happened was that 
The module was being re-imported after pytest.main was invoked. I inferred that because python was ignoring the dir(sys.modules[__name__]) check and re-initializing the config dictionary, and then I had lost any config values set from the command line and not read from the config file. If I don't invoke pytest.main the config dict is never re-initialized. 


If i am missing the point - could you provide a little reproducable example
and the pytest.main() invocations so that we can reproduce the problem?

[Pella,Chris]
As above... a main module which sets some global config dict values, and a config module where some more config dict values are read from a file. 
In the test file, if I have this...

import pytest
import config
 
class TestMe():
	def test_install_client(self):
		config.get('product')['client_build'] 

This fails because the 'product' key no longer exists in the global config dict. 
I can try and refactor and provide a small runnable example, if this isn't enough information. I know there is a workaround with pytest in that I can add another option to pytest itself using conftest.py... but I think this would be getting too messy and I would rather not have to do this for all the command line options we run the framework with:
def pytest_addoption(parser):
    parser.addoption('--product',action="store",default=None,
        help="specific Luna product")

def pytest_funcarg__product(request):
    return request.config.option.product


 
thanks,
holger

> For some reason, when pytest is in control the above test of the sys.modules dict for the "config" name fails and the config dictionary is reinitialized and my config state is lost.  Oddly, we tried something similar on a colleagues machine using a python 2.6 ( I am using 2.7) and we didn't have that problem.
> 
> Any ideas what could be happening? I find it a bit hard to debug because when pytest is in control on my machine it seems that I can't interact with the eclipse console after a breakpoint is hit.
> 
> Chris
> 
> 
> The information contained in this electronic mail transmission 
> may be privileged and confidential, and therefore, protected 
> from disclosure. If you have received this communication in 
> error, please notify us immediately by replying to this 
> message and deleting it from your computer without copying 
> or disclosing it.
> 
> 


> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python

The information contained in this electronic mail transmission 
may be privileged and confidential, and therefore, protected 
from disclosure. If you have received this communication in 
error, please notify us immediately by replying to this 
message and deleting it from your computer without copying 
or disclosing it.