[TIP] running tests from pytest.main

[holger krekel]

On Wed, Jul 04, 2012 at 13:56 -0400, Pella,Chris wrote:
> Delayed response... just got back from camping up the Ottawa river. Inline...
> 
> -----Original Message-----
> From: holger krekel [mailto:holger at merlinux.eu] 
> Sent: Tuesday, June 26, 2012 3:49 AM
> To: Pella,Chris
> Cc: testing-in-python at lists.idyll.org
> Subject: Re: [TIP] running tests from pytest.main
> 
> On Mon, Jun 25, 2012 at 14:52 -0400, Pella,Chris wrote:
> > We are invoking pytest using pytest.main from a custom functional
> > test framework we are layering on top of pytest. It creates a global
> > test configuration dictionary based on some command line switches
> > and some config file data.  I was trying to prevent the global
> > config dictionary from being re-initialized and making it a
> > singleton by doing this:
> > if "config" not in dir(sys.modules[__name__]):
> >                 config = {}
> >                 ....
> 
> Why are you going through the sys.modules indirection instead of just doing:
> 
>     # contained in some module
>     config = {}
> 
> ? 
> [Pella,Chris] I found it wasn't working in my application the way I
> would have thought. We are setting some values in the config
> dictionary from the command line because they are constantly changing,
> and some settings which are usually fixed on each machine are read
> from a file. I was just using the sys.modules as a check to keep
> python from reading the config file every time a test file imported
> the config module. 

There is a misunderstanding, i think.  The second import of a Python
module will not re-load a module, i.e. importing via:

    import xyz

and then in the same process from some other place:

    import xyz

will only load and initialize the module once.


> variables that were set from the command line were gone when pytest
> took over, and I thought this would show me if pytest was changing
> sys.modules, resulting in what I was seeing. 
> In the main framework module, we are setting some commandline options... I am just showing --product as an example.
> <in the main module>
> import config
> parser = OptionParser()
>     
> parser.add_option("--product",
>                       action = "store",
>                       type = "string",
>                       dest = "product",
>                       help = "select product under test")
>     
> config['product'] = options.product
> 
> <config module>
> # The config file has sections keyed by the 'product' option passed in the command line
> if 'config' not in dir(sys.modules[__name__]):
>     config = {}
>     #Command line settings set to reasonable defaults
>     #Config File settings
>     config_parser = ConfigParser()
>     config_parser.read("LunaConfig.ini")
>     config.update(dict(config_parser._sections))
>     
> <main test runner module>
> # Here is where we call pytest.main
> pytest_command = pytest_options
> if config.has_key('test_groups'):  
> 	pytest_command += "-v -m %s" %"".join(map(str,config.get('test_groups')))
> if config.has_key('file_to_test'):
>       pytest_command += " %s" %config.get('file_to_test')
> pytest.main(pytest_command)
> 
> If this is done in a module, "config" automatically is a singleton because
> all imported Python modules are singletons (i.e. a given module is only
> imported once usually).
> 
> [Pella,Chris] I stepped through the code using the pdb and found that what happened was that 
> The module was being re-imported after pytest.main was invoked. I inferred that because python was ignoring the dir(sys.modules[__name__]) check and re-initializing the config dictionary, and then I had lost any config values set from the command line and not read from the config file. If I don't invoke pytest.main the config dict is never re-initialized. 

However, there must be something going strange then.  I attach a file
"x.py" and "mod.py" and if you look at run "python x.py" then things
work as expected and not like you describe.  Can you provide a concrete
example that fails your expectations?

best,
holger

> 
> 
> If i am missing the point - could you provide a little reproducable example
> and the pytest.main() invocations so that we can reproduce the problem?
> 
> [Pella,Chris]
> As above... a main module which sets some global config dict values, and a config module where some more config dict values are read from a file. 
> In the test file, if I have this...
> 
> import pytest
> import config
>  
> class TestMe():
> 	def test_install_client(self):
> 		config.get('product')['client_build'] 
> 
> This fails because the 'product' key no longer exists in the global config dict. 
> I can try and refactor and provide a small runnable example, if this isn't enough information. I know there is a workaround with pytest in that I can add another option to pytest itself using conftest.py... but I think this would be getting too messy and I would rather not have to do this for all the command line options we run the framework with:
> def pytest_addoption(parser):
>     parser.addoption('--product',action="store",default=None,
>         help="specific Luna product")
> 
> def pytest_funcarg__product(request):
>     return request.config.option.product
> 
> 
>  
> thanks,
> holger
> 
> > For some reason, when pytest is in control the above test of the sys.modules dict for the "config" name fails and the config dictionary is reinitialized and my config state is lost.  Oddly, we tried something similar on a colleagues machine using a python 2.6 ( I am using 2.7) and we didn't have that problem.
> > 
> > Any ideas what could be happening? I find it a bit hard to debug because when pytest is in control on my machine it seems that I can't interact with the eclipse console after a breakpoint is hit.
> > 
> > Chris
> > 
> > 
> > The information contained in this electronic mail transmission 
> > may be privileged and confidential, and therefore, protected 
> > from disclosure. If you have received this communication in 
> > error, please notify us immediately by replying to this 
> > message and deleting it from your computer without copying 
> > or disclosing it.
> > 
> > 
> 
> 
> > _______________________________________________
> > testing-in-python mailing list
> > testing-in-python at lists.idyll.org
> > http://lists.idyll.org/listinfo/testing-in-python
> 
> The information contained in this electronic mail transmission 
> may be privileged and confidential, and therefore, protected 
> from disclosure. If you have received this communication in 
> error, please notify us immediately by replying to this 
> message and deleting it from your computer without copying 
> or disclosing it.
> 
> 
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: x.py
Type: text/x-python
Size: 165 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120705/5d4024c0/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mod.py
Type: text/x-python
Size: 13 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120705/5d4024c0/attachment-0001.py>