[TIP] unittest2 discovery / doctest integration?

[Jim Fulton]

On Wed, Apr 28, 2010 at 5:26 PM, Pete <pfein at pobox.com> wrote:
>
> On Apr 28, 2010, at 10:27 AM, Pete wrote:
>
>> Just wondering how to go about running doctests under unittest2's discovery... The approach mentioned here http://docs.python.org/dev/library/doctest.html#unittest-api gets the doctests run, but is kinda bogus (you get 2 sets of test runner output).  Glancing at unittest2's discovery stuff, it seems to look for TestCases, but not TestSuites. Just wondering if there's something obvious I'm missing, or if I should start poking through/ripping off doctest's implementation of integration (which is lightly documented).
>
> Appears the way to do this goes like so. In your mostly-unittests test module, define a load_tests:
>
> def load_tests(loader, tests, ignore):
>    tests.addTests(doctest.DocTestSuite(module_with_doctests))
>    return tests
>
> Pretty straightforward, though not at all obvious.  Looks like the documentation for doctest could use an update... http://docs.python.org/dev/library/doctest.html#unittest-api I'd write a patch (hello-to-my-friend-jesse-noller), but I'm not entirely sure what to say (tl;dr) - would replacing the code sample & changing its description to read "To integrate with unittest discovery, include the following in a module of unittests:" suffice?

Note that this isn't the only way to use a suite and load_tests came
along much later than the doctest integration.

The unittest docs say: "For the ease of running tests, as we will see
later, it is a good idea to provide in each test module a callable
object that returns a pre-built test suite."  This is the advice we
took when we designed the Zope test runner, which focuses on
discovering the test suite function, rather than individual test
cases. In general, this led to our tests being fairly suite centric,
which explains why I integrated doctest the way I did. <shrug>

> As for Jim's comment re: "I wish I'd used TestCases instead of TestSuites for doctest integration"... I'm gonna have to disagree.  The problem is that there can be multiple doctests->TestCases generated by DocTestSuite/DocFileSuite.  As discovery currently works by inspecting the module dict, there's not good way to bind those to multiple names.  Arguably, discovery should look for TestSuites as well...

Talking about unittest is rather hard because there are so many ways
to use it and the docs are rather challenging.  When we create
TestCase classes, they typycally define many methods that are run as
individual tests. I have no idea whether these become individual test
cases or not, and I'm too lazy to look. :) The various test finders have
no trouble finding these classes and doing the right thing with
them. It would be easy enough to create a TestCase subclass that
handled multiple doc tests in much the same way that TestCase
handles multiple test methods.

If I was thinking in a test-case centric rather than suite-centric
way, I'm sure that I could come up with a different API that handled
multiple doctests just fine.

Jim

--
Jim Fulton