[TIP] nose2 backwards compatibility questions

[jason pellerin]

Sorry I wasn't able to jump back in yesterday, hectic workday.

On Tue, Aug 10, 2010 at 2:17 PM, Michael Foord
<fuzzyman at voidspace.org.uk> wrote:
> On 10/08/2010 14:40, Michael Foord wrote:
>>
>> On 09/08/2010 17:49, jason pellerin wrote:
>>>
>>> Some questions for nose2-philes.
>>>
>>> 1) How much do you care about nose-style test discovery? nose does a
>>> bunch of extra work to support real-world package layouts, such as
>>> prepending ./lib and ./src to sys.path
>>
>> This you should still be able to do without issue, right?
>
> Ah no, it requires continuing discovery into these directories - with them
> as the top level directory - even though they don't contain an __init__.py.
> I will propose a fix to the TestLoader that allows this to work.

Sounds good. nose's pattern is to actually start discovery in those
directories, if they are present, to ensure that if the project layout
is something like:

lib/mod.py
tests/tests.py

that mod is importable from tests.

>>
>>> and looking in test* dirs even
>>> when they aren't packages.
>>
>> Do you do this recursively - or just at the top level?
>>
>
> For this I still want to understand what is allowed (and how it works) in
> order to either allow it to be done through a plugin or making it possible
> in the standard test discovery machinery.

This gets into the part of nose's discovery that gets hairy, the part
I think Kumar is suggesting leaving on the cutting room floor. nose
will look for tests in any directory that looks like 'test*' (more or
less), whether that directory is a package or is not a package, inside
of a package or out. To support project layouts where the tests are
organized into directories that are not packages, nose's importer
always considers the full path to a module, and if the new full path
differs from the full path of a module of the same name already in
sys.modules, the old module will be evicted in favor of the new. This
works only because nose does its discovery lazily -- nose's
loadTestsFromDir method is a generator. So it loads and runs all of
the tests in dir X, then proceeds to load and run all of the tests in
dir Y. Obviously, unittest2 doesn't do that and I don't expect that it
will, so that makes the importer question largely moot, and limits the
kinds of test layouts that nose2 can support to, I think:

- The unittest2 standard, which I think we should encourage as a best
practice, a test* package inside the main package
- One test* directory at the top level of the project. Subdirectories
allowed but without special importer tricks, so no support for, eg:

tests/test_foo/tests.py
tests/test_bar/tests.py

... unless tests.test_foo and tests.test_bar are packages.

I'm ok with dropping support for the "test dir (that is not itself a
package) inside a package" layout, and dropping the custom importer.
So I think all that's needed is a way for a plugin to tell the test
loader to descend into more directories in a particular order, as I
described above.

JP