[TIP] Doctest or unitest?
Michael Foord
fuzzyman at voidspace.org.uk
Wed Mar 7 13:01:30 PST 2007
Benji York wrote:
> Michael Foord wrote:
>> If you had lots of nasty edge case testing which didn't fit well into
>> documentation then I guess you could put it into a separate file.
>
> Either a separate file, or if there aren't too many, footnotes. (I
> added an extension to doctest to recognize ReST footnotes and execute
> them at the point of reference instead of at the point of definition.
> They're good for getting corner cases and repeated setup code out of
> the narrative flow of the document).
>
>> Isn't typing '...' all the time annoying ? (or do you code first,
>> test later... ? ;-)
>
> I try to not use ellipsis myself. We generally now use something
> called the "ReNormalizer" that lets you create regular expressions
> that replace matches with dummy values for those sorts of things. For
> example, if you wanted to look at the repr of something and had to
> deal with the object's ID changing with each run, with ellipsis you'd
> do this:
>
> >>> foo
> <Foo object at ...>
>
> With the ReNormalizer you could make a substitution that replaces all
> the addresses with something that looks like an address, but doesn't
> vary and your test would just look like this:
>
> >>> foo
> <Foo object at 0x12345678>
>
> That way a reader's internal Python-output-parser would still be happy
> but the example test would remain deterministic.
>
What about the ellipsis needed for indented blocks - there seemed to be
plenty in the example you pointed me to.
>> Don't you also lose the help of the IDE with indentation and
>> autocomplete because you are no longer writing the tests in Python
>> files ?
>
> No one I know uses an IDE :)
>
You should try the autocomplete and source browsing features of Wing
(Eclipse / PyDev is pretty good as well).
Control-click taking you to definition is invaluable as well. We also
have Wing scripted so that we can run the unit tests associated with any
file at the push of a button.
> That being said, I personally use a very simple Vim mode that
> recognizes Python code in a doctest and highlights/completes/whatever
> appropriately. I would assume that's pretty easy to do in other
> editors/IDEs.
>
>> We are writing an application (rather than a library), and so don't
>> *want* to document all of our internal classes / modules - so
>> documentation driven design has little to offer us for these tasks.
>> (and so the idea of using doctests becomes even weirder.)
>
> I can't agree there, but don't want to burden you with a long,
> wandering argument that I don't have time to make into a short,
> compelling one. :)
>
>> For our acceptance (functional) tests it might be quite a nice way of
>> doing them - we'd have to make our automation API look even more like
>> a DSL though for it to be readable.
>
> That was exactly the motivation for testbrowser (a testing focused,
> programmatic web user agent). We had a way of doing functional tests,
> but the experience of reading the tests was very different from the
> experience of a user interacting with the app. If our experience with
> testbrowser is any indication, developing a good test API for your app
> will go a long way to making testing easier and also help find more bugs.
We have a very good automatic test API - but of course using a framework
built on top of unittest.
More to the point we are using a very different methodology than you.
You are doing documentation driven design (which is fine), we are doing XP.
Part of XP is that you *don't* create unnecessary documentation,
although you write user documentation when the customer prioritises it.
The code we write *must* be readable - if it isn't it needs refactoring.
The tests *must* be readable (and I like to think that ours are
generally pretty good). The unit tests act as specification and
documentation for the code.
This is a very different style of development, but again it is my
understanding that it is pretty orthodox XP.
It is assisted by the fact that we are a geographically non-dispersed
team. We pair on all our production code. One team member will 'own' a
user story, but we swap who pairs with user story owners *every* day.
That way we all get to work on the whole code base and knowledge is
spread rapidly.
Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml
More information about the testing-in-python
mailing list