[TIP] Doctest or unitest?

[Jim Fulton]

On Mar 3, 2007, at 12:29 PM, Grig Gheorghiu wrote:

> This is the question that Michael Foord asks in his latest blog post:
>
> http://www.voidspace.org.uk/python/weblog/ 
> arch_d7_2007_03_03.shtml#e652

I commented on this.

>
> To me, they represent 2 different styles of testing: procedural
> (unittest) vs. declarative (doctest).

I don't see doctests as particularly declarative.

For me, the two styles are: "framework heavy" vs "example based".

I sometimes refer to the former as "javiotic". :)


> I had an aha-type moment during
> PyCon when I realized that both Fit/Fitnesse and doctest are very
> similar in that they represent 'example-driven testing' (I didn't
> invent the term example-driven, Brian Marick did). You specify inputs
> with expected outputs, in the form of examples.

Yes.

I'll add that they also support a style of development that I call  
"documentation driven".  If done well. this encourages you to  
document how the user will use your library or application to achieve  
their goals. I find that this viewpoint allows problems in interfaces  
(API or user) to be discovered or avoided early.

> With unittest, and the other frameworks based on it or similar to it,
> there's more procedural stuff: do this, do that, then assert that it
> happened (FitNesse also supports this with DoFixture fixtures).

I find the testing framework calls to be mostly pointless and detract  
from understanding what a test is actually testing.

Something to keep in mind about writing automated tests is that it is  
a form of meta programming. You are writing code about other code.  
This makes the activity a lot more complex and it makes reading tests  
potentially a lot harder.  When you read normal code, the intent of  
the code is usually known.  With tests, the intent is far less  
obvious.  I find that doctests tend to make test code more  
understandable because:

- They don't litter the code with superfluous testing framework calls,

- They make it easier to write prose that code.  Some people defend  
unittest-style tests by telling me that it is easy to write comments  
in unittest-based unit tests, but I find this rarely happens in  
practice.

- doctests force you to go further out of your way to create  
"helpful" abstractions that allow code sharing between tests and that  
make reading tests much more difficult.

Of course, doctests don't prevent inscrutable tests.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org