[TIP] PyTest test directory structure confusion

[Marius Gedminas]

Hi!

On Tue, Aug 27, 2019 at 09:45:27PM -0700, Tony Cappellini wrote:
> I’m new to using PyTest. I’ve got a very simple python file, and a very simple
> test file for it.
...

> I’ve been having a problem getting a unittest to be able to access the file
> it’s trying to test. I’m sure that I don’t have the correct directory
> configuration, but at the same time, I’m not sure what the correct directory
> configuration should be though.

You have the simplest possible case: one code file, one test file.  You
don't need a directory structure at this point, just the two files:

    mycode.py
    tests.py

inside your tests.py you can do

    import mycode

    def test_my_stuff():
        assert mycode.stuff(42) == 25   # etc.

and you should be able to run the tests with

    pytest tests.py

If you don't want to keep repeating 'tests.py' on the command line,
create a pytest.ini with

    [pytest]
    testpaths = tests.py

and then you can run just

    pytest

That's it!

I use this structure for a few of my small Python projects, e.g.
https://github.com/mgedmin/ghcloneall


> Page 25 of “Python Testing with PyTest refers to __init__.py files in the same
> directory as the test files themselves.
> 
> Under Project Structure, This website 
> [1]https://automationpanda.com/2017/03/14/python-testing-101-pytest/
> 
> states that __init__.py files should NOT be in the same directory as the test
> files.

I've no idea why that page makes this claim.  I have my tests in a
package in https://github.com/mgedmin/pyspacewar, and pytest has no
trouble with that structure.

> __init__.py files have always been the biggest headache for me, as I’ve never
> been able to find a clear reference as to what should or shouldn’t be in those
> files.
> Why is it that some are empty, while others have tons of imports?

I assume you know what they're for?  If not, see the Python Tutorial
section on packages: https://docs.python.org/3/tutorial/modules.html#packages

As for the contents, the short answer is: if you're not sure, keep
__init__.py files empty.  But if you feel like you want to put some code
in there, there likely won't be any harm.

> I would appreciate if someone could provide a reference as to how a PyTest
> project should be structured, where __init__ files should and shouldn’t be,
> along with their contents (if any).

There's more than one common source layout, and which is best is a
matter of opinion.


The one I'm used to has a 'src/' directory at the top, and puts all the
tests as subpackages inside the actual code tree.  Like, imagine if the
example I gave before grew and grew and became too big for a single
source file:

    mycode.py
    tests.py
    pytest.ini
    setup.py
    tox.ini

I've added a setup.py because you probably want to be able to pip
install the thing, and pull it any dependencies it has.  I've added a
tox.ini because tox is a wonderful tool for automating the tests -- it
creates virtualenvs for you so you get all the code and test
dependencies without polluting your global Python installation and
interfering with other projects.

(For completeness, here's what a setup.py looks:

    from setuptools import setup
    setup(
        name='mycode',
        version=...,
        author=...,
        license=...,
        url=...,
        description=...,
        long_description=...,
        py_modules=['mycode'],
    )

and here's tox.ini:

    [tox]
    envlist = py37

    [testenv]
    deps = pytest
    commands = pytest {posargs}

)

So, at this point splitting the code into several modules will result in

    src/
        mycode/
            __init__.py                    # maybe empty, maybe not
            things.py
            more_things.py
            tests/
                __init__.py                # empty file
                test_mycode.py
                test_things.py
                test_more_things.py
    pytest.ini
    setup.py
    tox.ini

Here each code module has a corresponding test module, for my own
convenience: when I want to find the unit tests for a function
mycode.things.do_x(), I'll know to look in mycode.tests.test_things
for test functions called test_do_x (with optional suffixes if there's
more than one test).

To make this work, pytest.ini changes from

    [pytest]
    testpaths = tests.py

to

    [pytest]
    testpaths = src

and you can't run 'pytest' without arguments any more (because ./src is
not in PYTHONPATH -- this is the downside of using a top-level 'src'
subdirectory), but you can run 'tox' without arguments.  tox.ini is
unchanged, setup.py changes from using

    from setuptools import setup
    setup(
        ...
        py_modules='mycode',
    )

to

    from setuptools import setup, find_packages
    setup(
        ...
        packages=find_packages('src'),
        package_dir={'': 'src'},
    )

Now, if you want to continue to write code like

    from mycode import stuff

then mycode/__init__.py will have to define the 'stuff' function, or
import and re-export it like this:

    # mycode/__init__.py
    from .things import stuff   # reexport

But if you're fine requiring that all users of mycode change their code
to do

    from mycode.things import stuff

then mycode/__init__.py can be an empty file.


Hope that helped!

Marius Gedminas
-- 
/*
 * The lockdep graph lock isn't locked while we expect it to
 * be, we're confused now, bye!
 */
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20190828/81cb52f6/attachment.pgp>