[TIP] Declarative style acceptance tests (was: Using doctest for functional tests / user stories)

[Barry Warsaw]

On Jan 06, 2014, at 01:48 PM, Chris Jerdonek wrote:

>In my experience (and I'm sure others would agree), doctests are less
>useful the more complicated your tests are.

I love doctests and use them extensively in Mailman and other projects.  I do
think there are strategies for using them effectively though, part of which
involves recognizing that they do not replace unittests but augment them.

Several years ago I went through some experiments to *only* use doctests,
which was very instructive because I learned where (IMHO) they do not work
very well.  For example, the emphasis should be on the "doc" part, meaning
that doctests are great when documenting how to use a feature or an API,
especially when used in combination with Sphinx for generated output.

These days I write all my documentation using doctest/reST, but I try to stick
to good paths and recommended usage.  Doctests don't work as well for covering
all code paths, and while I occasionally document tracebacks, I think testing
the bad paths in your code tend to work better in unittests.

Also, doctests out of the box need some help.  I have my own hacks on top of
stdlib, and I've been meaning to look at dutest for a while.

Here's Mailman's doctest integration with nose2:

    http://tinyurl.com/l2trn4a

and my setup of globals for doctests (which I really try to keep to a minimum
- it's a balance between magic and readability):

    http://tinyurl.com/k5ecy2u

Here is a pretty good example doctest[1].

    http://tinyurl.com/pbx2kqw

-Barry

[1] The transaction.commit() and .aborts() are a bit ugly, but necessary due
to the fact that the REST server runs in a separate process.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20140106/c8e5924c/attachment-0001.pgp>