[TIP] Doctest or unitest?

[Michael Foord]

Jim - can you (or someone else) point me to an example source file which 
uses doctest for unit testing.

I remain deeply unconvinced (cool ReST hacks aside...), but intrigued, 
and I'd like to see some examples.

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Jim Fulton wrote:
> On Mar 5, 2007, at 7:42 PM, Andrew Bennetts wrote:
>
>   
>> Jim Fulton wrote:
>> [...]
>>     
>>>> - Certain operations are simply more awkward in a doctest.   
>>>> Reusing  common
>>>> set up and tear down is harder (you can't just subclass or  
>>>> import  from a
>>>> doctest), so people tend to just keep adding to the existing  
>>>> doctest file
>>>> rather than making a new file when they should.  Again the result  
>>>> is long,
>>>> rambling doctest files that are both poor tests and poor  
>>>> documentation.
>>>>         
>>> This certainly doesn't need to be the case.  I usually do test setup
>>> in Python.  I use the unittest integration of doctests and you can
>>> pass set-up and tear-down functions to this interface.  I generally
>>> avoid setup in the text unless it adds to the story.
>>>       
>> But this is more awkward than doing so in a TestCase class.  You  
>> have to edit
>> multiple files to arrange this.  The reader can't simply scroll  
>> within the file
>> to see what the assumed context (i.e. setUp) is for the code  
>> examples to
>> actually work.
>>     
>
> In practice, I've seen a similar problem with TestCase based tests.   
> You end up searching a bunch of base classes, trying to piece  
> together what the context was from lots of different places.  I find  
> that TestCase tests encourage abstraction that tends to be very harmful.
>
>   
>>   If I want to base a new test off an existing one, I have to
>> randomly grep around until I find where the DocTestSuite is  
>> constructed and
>> chase where the setUp is defined, rather than knowing that I can  
>> simply extract
>> the setUp method right in front of me into a common base class.
>>     
>
> I guess that this depends on how you construct your tests.  Our  
> packages have a single tests module that defines or sets up all of  
> the tests in that package. If you find a text file, you know to look  
> for a tests file in the same package.  (Note that this is *current  
> practice.) I haven't found finding the setup to be that big a deal.
>
> Note that I find that there are two kinds of setup code, code that is  
> interesting to the reader, and code that isn't.  Obviously, the  
> former should be included in the documentation itself.  We've  
> developed an extension to doctest that allows us to put some setup  
> and edge case code that *might* be interesting to the user in ReST  
> footnotes.  This lets us deal with borderline cases and also lets us  
> reset the world conveniently many times in a doctest, since the  
> footnoted code is run for every footnote reference. (We really need  
> to make this extension more widely available.)
>
>   
>> As a result, I've seen people simply choose to extend an existing  
>> doctest file
>> because figuring out how to make a new one was too much hassle.
>>     
>
> Yup
>
>   
>> This isn't a huge problem, but I do think it's fair to say that  
>> doctests are
>> a bit more awkward to work with in this respect than other tests.
>>     
>
> I understand and appreciate your point, however, I prefer the  
> situation in doctest for myself because it is so restricted and  
> discourages the sort of abstractions encouraged by TestCase that I  
> find usually makes TestCase tests hard to follow.
>
>   
>>> I haven't found this to be an issue myself.  This could just be a
>>> matter of usage patterns. WRT line numbers, our test failure outputs
>>> do in fact produce correct line numbers. I don't know why your's
>>> aren't. Also, doctests do have names that can be used for test
>>> selection.  Of course, you can only select the entire text body.
>>>       
>> With regards to line numbers, I think maybe that bug has been fixed  
>> now (maybe
>> in python 2.4?).
>>     
>
> Ah, probably.  The file-based doctests in 2.3 might have had that  
> problem.  While we still do development with Python 2.3, we use the  
> Python 2.4 based doctests in 2.3. This led us to make a copy of the  
> 2.4 doctest module in zope.testing, which now means we've forled  
> doctest. :(  We need to fix this. Having packages in the standard  
> library is a pain.  It would have been easier to make the doctest  
> fixes widely available sooner if doctest was distributed separately.  
> Bit that's a different controversy. :)
>
> Jim
>
> --
> Jim Fulton			mailto:jim at zope.com		Python Powered!
> CTO 				(540) 361-1714			http://www.python.org
> Zope Corporation	http://www.zope.com		http://www.zope.org
>
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>