[TIP] pytest: mocking module imports

holger krekel holger at merlinux.eu
Sun Apr 19 00:58:40 PDT 2015


Hi Vinicius,

On Fri, Apr 17, 2015 at 18:21 -0300, Vinicius Kwiecien Ruoso wrote:
> Hi Roy!
> 
> In this particular case I think a delegate design does not scale, as
> in the end the class inherit from dict (having to implement lots of
> methods), and I will have lots of those classes. Thinking a little
> better, this class could be in the libA package, as it is in a "utils"
> package in libB that does not use it.
> 
> But regardless of that, the problem still the same. I need to mock the
> module import by changing the sys.modules dictionary because the mock
> library also tries to import the module to be mocked (this will happen
> with other objects that are not necessarily used for inheritance).
> That way the import statement will work, and a mock object will be
> returned.
> 
> What I'm really looking for is the correct 'hook' that would allow me
> to run code (this sys.modules mock) before the test_* files are
> imported by py.test. I understand that there might not be a way to do
> this from the design of py.test (collect tests first then execute), so
> I'm speculating. I don't fell good by keeping this mocking code at the
> module level in conftest.py files, as it can be quite confusing, and I
> can't clean up after execution.

There is one hook which you can use from a setuptools plugin which is
called before any conftest.py or user code is imported,
pytest_load_initial_conftests.  It is used by pytest-cov and other
plugins to get a foot in the door very early on.  This plugin
could be called "pytest-mock_libb" or so.  But it's not an elegant
or good solution i think.

It's better to just aim for a simple root-level conftest.py file
which does not globally import anything that triggers importing of the
library you want to mock.  And then do early mocking of modules by
creating module objects and putting them into sys.modules.  

However, it's best to avoid library-importing also during test module
collection and rather put all relevant imports into fixture functions or
tests, because you could then use ("autouse" or plain) fixtures to do
the mocking.  The latter is what i would aim for.  When mocking you can guard 
against accidental mistakes by checking first that the library wasn't
already imported by checking non-existence in sys.modules.

best,
holger



> Thanks!
> Vinicius
> 
> 2015-04-17 17:18 GMT-03:00 Roy Wright <roy at wright.org>:
> > Maybe use a delegate pattern instead of inheritance?  This would allow runtime binding of the parent.
> >
> > Another thought is to put ClassY in a config file then dynamically resolve it for the ClassX definition.
> >
> > HTH
> >
> > A free society is a single class society where everyone has the same rights.
> >
> > On Apr 17, 2015, at 2:29 PM, Vinicius Kwiecien Ruoso <vinicius.ruoso at gmail.com> wrote:
> >
> >> Sorry. The attachment follows.
> >> (and consider that test_classa_instance function does not have the
> >> mock_classB parameter at the second example).
> >>
> >> 2015-04-17 16:23 GMT-03:00 Vinicius Kwiecien Ruoso <vinicius.ruoso at gmail.com>:
> >>> Hi!
> >>>
> >>> Thanks for your answer. Unfortunately, this does not work for me.
> >>> libB is not importable during the test, and at some point the test try
> >>> to import it.
> >>>
> >>> Let me show a real minimal example:
> >>> ├── src
> >>> │   └── mylib
> >>> │       ├── __init__.py
> >>> │       ├── liba.py
> >>> └── test
> >>>    └── mylib
> >>>        ├── conftest.py
> >>>        └── test_liba.py
> >>>
> >>> 8<---------
> >>> <liba.py>
> >>> from mylibB.libb import ClassB
> >>>
> >>> class ClassA(ClassB):
> >>>    pass
> >>>
> >>> <conftest.py>
> >>> import os
> >>> import sys
> >>>
> >>> _src_path = os.path.join(os.path.dirname(__file__), '..', '..', 'src')
> >>> sys.path.insert(0, os.path.abspath(_src_path))
> >>>
> >>> <test_liba.py>
> >>> from mock import patch
> >>>
> >>> @patch('mylibB.libb.ClassB')
> >>> def test_classa_instance(mock_classB):
> >>>    print mock_classB
> >>> 8<---------
> >>>
> >>> This fails with: "ImportError: No module named mylibB" while executing
> >>> the patch code.
> >>>
> >>> My real intent on the test_liba.py file was (and for that to work, I
> >>> need to mock libb module import at the conftest.py file):
> >>>
> >>> import mylib.liba
> >>>
> >>> def test_classa_instance(mock_classB):
> >>>    print mylib.liba.ClassA()
> >>>
> >>>
> >>> Attached you may find this file structure if you want to run it.
> >>>
> >>> Just as a note, I'm using Python 2.6 right now. I intend to support
> >>> Python 2.7, but not 3.x at this time.
> >>>
> >>> Thanks again!
> >>> Vinicius
> >>>
> >>> 2015-04-17 14:47 GMT-03:00 Zhi An Ng <ngzhian at gmail.com>:
> >>>> Hi,
> >>>> I believe you can mock out ClassY as such:
> >>>>
> >>>> @patch('liB.ClassY')
> >>>> def test_unit_test(self):
> >>>>    stuff
> >>>>
> >>>>
> >>>> More info:
> >>>> https://docs.python.org/3.3/library/unittest.mock.html#quick-guide
> >>>>
> >>>> Best,
> >>>> Zhi An
> >>>>
> >>>> On Fri, Apr 17, 2015 at 1:35 PM, Vinicius Kwiecien Ruoso
> >>>> <vinicius.ruoso at gmail.com> wrote:
> >>>>>
> >>>>> Hi!
> >>>>>
> >>>>> I've been using pytest and it is great. There is just one thing that
> >>>>> I'm not satisfied with the way I've implemented, that is module level
> >>>>> moking.
> >>>>>
> >>>>> My scenario is as follows: libA and libB where libA depends on libB
> >>>>> (uses class definition as superclass). Let's assume this code:
> >>>>>
> >>>>> <libA.py>
> >>>>> from libB import ClassY
> >>>>>
> >>>>> class ClassX(ClassY):
> >>>>>   pass
> >>>>>
> >>>>> <libB.py>
> >>>>> class ClassY(object):
> >>>>>   pass
> >>>>>
> >>>>> Those modules live on different packages/repositories. LibB can be
> >>>>> unit tested just fine, but we need to mock libB module to allow libA
> >>>>> to be unit tested (libB won't be available in the moment of the test).
> >>>>>
> >>>>> In a conftest.py file in the tests directory of libA, I've managed to
> >>>>> mock the module import by mocking sys.modules dictionary, but I did
> >>>>> this at the module (global) scope. I was not able to do it with any
> >>>>> hook (like pytest_configure), because the tests files are imported
> >>>>> before the hook is called. So a test_libA.py file with "import libA"
> >>>>> would cause the test to fail.
> >>>>>
> >>>>> Am I doing something wrong or is this really the way to go?
> >>>>>
> >>>>> Greetings,
> >>>>> Vinicius
> >>>>>
> >>>>> _______________________________________________
> >>>>> testing-in-python mailing list
> >>>>> testing-in-python at lists.idyll.org
> >>>>> http://lists.idyll.org/listinfo/testing-in-python
> >> <minimal-test.tar>
> >> _______________________________________________
> >> testing-in-python mailing list
> >> testing-in-python at lists.idyll.org
> >> http://lists.idyll.org/listinfo/testing-in-python
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python

-- 
about me:    http://holgerkrekel.net/about-me/
contracting: http://merlinux.eu



More information about the testing-in-python mailing list