[TIP] including test files when running tox

[Chris Jerdonek]

Thanks very much for your reply, Holger.  Tox is quite awesome, by the
way! :)  I just got it working for my project with Python 2.4 through
3.2.

See replies below.

On Tue, Apr 10, 2012 at 9:03 PM, holger krekel <holger at merlinux.eu> wrote:
> On Tue, Apr 10, 2012 at 10:45 -0700, Chris Jerdonek wrote:
>> I am new to tox and have a question.
>>
>> My python project has files needed for testing that are not included
>> when running (for example) "pip install".  These include things like
>> test data files outside the main package, sub-packages of the main
>> package with modules containing unit tests, and packages outside the
>> main package including python modules with test data.
>>
>> What is the recommended way to ensure that test-related files like
>> these are available (and in the proper relative location) when running
>> tox?
>
> it probably depends a bit on the test runner you are using.
>
> data files: If you can instrument your test run by specifying data files
> etc. on the command line then you can use tox's variables to point to
> specific files/dirs.

Can you say more?  And are you suggesting that tox not copy the files
but that absolute paths be used instead?  I will say more about the
data files below when I point you to the specific project.  If tox
doesn't copy the files, it seems like it will become more difficult
for Distribute's `test` command to convert from Python 2 to 3 those
test files that are also Python modules.

> unittests-as-subpackages-of-main-packages: some people argue that such tests should be installed in which case you can just run them.

I would be willing to go this route (which I've already done with some
of the test files).  However, can you provide me a link/reference for
this point?  I would like to have a better understanding of the
argument/rationale.

> python modules with test data: not sure i understand - but with tox you
> can just run them as part of your test commands.

My project has an "examples" directory next to the package directory
that contains python modules which are used in some tests.  Think of
these as data files for some tests which take the form of (importable)
Python modules.  Below I provide a link to the project itself.  You
can take a look if you'd like.

> Now, the interaction with 2to3 complicates the picture.  I guess a more concrete example and the result of some trying to do things from your side might allow for more concrete suggestions.

As I said below, I chose to go the route of using Distribute's `test`
command (with the "use_2to3" keyword) to do the 2to3 conversion.  I
was able to get all of this working.  My only question at this point
is how to take a more "correct" approach to achieving my aims.  The
current approach (which I describe below) seems like a hack in some
ways.

My project as of now is here:

https://github.com/defunkt/pystache/tree/302c806ecd296dafdcd449350b001040c27b86c9

See, in particular, the setup.py and tox.ini files.  As you can see, I
was able to get the test data files and modules outside of the package
directory to copy over with tox by using the "package_data" keyword
argument to setup() in conjunction with the parent directory prefix:

package_data = {
    # Include the README so doctests can be run.
    # TODO: is there a better way to include the README?
    'pystache': [
        '../README.rst',
        '../ext/spec/specs/*.json',
        '../ext/spec/specs/*.yml',
    ],
    # Include template files so tests can be run.
    'examples': template_files,
    'pystache.tests.data': template_files,
    'pystache.tests.data.locator': template_files,
},

However, this is somewhat of a hack because, for example, the README
file (which contains doctests) and the ext and examples directories
(which contain test data) wind up in the site-packages directory
alongside the main package directory pystache.  This could
theoretically cause collisions with other projects.

Maybe one solution would be to include these directories within the
pystache package directory itself.  However, it's not clear to me when
test data should be included as part of an installation, and when it
should be included only as part of a source distribution.

Thanks,
--Chris

>
> best,
> holger
>
>> I'm using Distribute's `test` command to handle any conversion from
>> Python 2 to 3 (as described here)--
>>
>> http://packages.python.org/distribute/python3.html
>>
>> so it won't be necessary for tox to do any of those conversions.
>>
>> Thanks,
>> --Chris
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>