[TIP] Doctest or unitest?

Michael Foord fuzzyman at voidspace.org.uk
Wed Mar 7 13:01:30 PST 2007


Benji York wrote:
> Michael Foord wrote:
>> If you had lots of nasty edge case testing which didn't fit well into 
>> documentation then I guess you could put it into a separate file.
>
> Either a separate file, or if there aren't too many, footnotes. (I 
> added an extension to doctest to recognize ReST footnotes and execute 
> them at the point of reference instead of at the point of definition.  
> They're good for getting corner cases and repeated setup code out of 
> the narrative flow of the document).
>
>> Isn't typing '...' all the time annoying ? (or do you code first, 
>> test later... ? ;-)
>
> I try to not use ellipsis myself.  We generally now use something 
> called the "ReNormalizer" that lets you create regular expressions 
> that replace matches with dummy values for those sorts of things.  For 
> example, if you wanted to look at the repr of something and had to 
> deal with the object's ID changing with each run, with ellipsis you'd 
> do this:
>
> >>> foo
> <Foo object at ...>
>
> With the ReNormalizer you could make a substitution that replaces all 
> the addresses with something that looks like an address, but doesn't 
> vary and your test would just look like this:
>
> >>> foo
> <Foo object at 0x12345678>
>
> That way a reader's internal Python-output-parser would still be happy 
> but the example test would remain deterministic.
>
What about the ellipsis needed for indented blocks - there seemed to be 
plenty in the example you pointed me to.


>> Don't you also lose the help of the IDE with indentation and 
>> autocomplete because you are no longer writing the tests in Python 
>> files ?
>
> No one I know uses an IDE :)
>

You should try the autocomplete and source browsing features of Wing 
(Eclipse / PyDev is pretty good as well).

Control-click taking you to definition is invaluable as well. We also 
have Wing scripted so that we can run the unit tests associated with any 
file at the push of a button.

> That being said, I personally use a very simple Vim mode that 
> recognizes Python code in a doctest and highlights/completes/whatever 
> appropriately.  I would assume that's pretty easy to do in other 
> editors/IDEs.
>
>> We are writing an application (rather than a library), and so don't 
>> *want* to document all of our internal classes / modules - so 
>> documentation driven design has little to offer us for these tasks. 
>> (and so the idea of using doctests becomes even weirder.)
>
> I can't agree there, but don't want to burden you with a long, 
> wandering argument that I don't have time to make into a short, 
> compelling one. :)
>
>> For our acceptance (functional) tests it might be quite a nice way of 
>> doing them - we'd have to make our automation API look even more like 
>> a DSL though for it to be readable.
>
> That was exactly the motivation for testbrowser (a testing focused, 
> programmatic web user agent).  We had a way of doing functional tests, 
> but the experience of reading the tests was very different from the 
> experience of a user interacting with the app.  If our experience with 
> testbrowser is any indication, developing a good test API for your app 
> will go a long way to making testing easier and also help find more bugs.
We have a very good automatic test API - but of course using a framework 
built on top of unittest.

More to the point we are using a very different methodology than you.

You are doing documentation driven design (which is fine), we are doing XP.

Part of XP is that you *don't* create unnecessary documentation, 
although you write user documentation  when the customer prioritises it.

The code we write *must* be readable - if it isn't it needs refactoring.

The tests *must* be readable (and I like to think that ours are 
generally pretty good). The unit tests act as specification and 
documentation for the code.

This is a very different style of development, but again it is my 
understanding that it is pretty orthodox XP.

It is assisted by the fact that we are a geographically non-dispersed 
team. We pair on all our production code. One team member will 'own' a 
user story, but we swap who pairs with user story owners *every* day.

That way we all get to work on the whole code base and knowledge is 
spread rapidly.

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml



More information about the testing-in-python mailing list