[TIP] Doctest or unitest?

[Andrew Bennetts]

Jim Fulton wrote:
[...]
> >
> >- Certain operations are simply more awkward in a doctest.  Reusing  common
> >set up and tear down is harder (you can't just subclass or import  from a
> >doctest), so people tend to just keep adding to the existing doctest file
> >rather than making a new file when they should.  Again the result is long,
> >rambling doctest files that are both poor tests and poor documentation.
> 
> This certainly doesn't need to be the case.  I usually do test setup  
> in Python.  I use the unittest integration of doctests and you can  
> pass set-up and tear-down functions to this interface.  I generally  
> avoid setup in the text unless it adds to the story.

But this is more awkward than doing so in a TestCase class.  You have to edit
multiple files to arrange this.  The reader can't simply scroll within the file
to see what the assumed context (i.e. setUp) is for the code examples to
actually work.  If I want to base a new test off an existing one, I have to
randomly grep around until I find where the DocTestSuite is constructed and
chase where the setUp is defined, rather than knowing that I can simply extract
the setUp method right in front of me into a common base class.

As a result, I've seen people simply choose to extend an existing doctest file
because figuring out how to make a new one was too much hassle.

This isn't a huge problem, but I do think it's fair to say that doctests are
a bit more awkward to work with in this respect than other tests.

> I haven't found this to be an issue myself.  This could just be a  
> matter of usage patterns. WRT line numbers, our test failure outputs  
> do in fact produce correct line numbers. I don't know why your's  
> aren't. Also, doctests do have names that can be used for test  
> selection.  Of course, you can only select the entire text body.

With regards to line numbers, I think maybe that bug has been fixed now (maybe
in python 2.4?).

-Andrew.