[TIP] On using doctests all the time WAS: Python Testing book review

[Olemis Lang]

New thread to leave the other just for the book review talk ;o)

On Mon, Mar 8, 2010 at 9:39 PM, Andrew Bennetts <andrew at bemusement.org> wrote:
> Olemis Lang wrote:
>> On Sat, Mar 6, 2010 at 1:36 PM, Grig Gheorghiu <grig.gheorghiu at gmail.com> wrote:
>> > I think you'll find broken code with typos in most technical books.
>>
>> Unless the book itself relies on self-tested docs (and that's an
>> important use case for all those people who lost their mind and use
>> `doctest` ... even if some think they should say «help me, I have
>> written a test suite using nothing but string matching assertions,
>> what is wrong with me?!» ) . AFAICR that's how Hg book is written (and
>> IMHO that was a great decision ;o)
>
> FWIW, I'm the person you quote who said (tongue-in-cheek)

I know

;o)

> I think automatically tested documentation is a wonderful idea.

Cool. IMO automatically tested documentation is not just for writing
books, but also to document the code ;o)

>
> This is an orthogonal concern to whether a good way to write precise and
> maintainable assertions is by turning every single outcome to assert
> into a string and then comparing it to another string.

Good point . I think that, as long as True be printed as True and
False printed as False there's almost no difference between writing
doctests and writing unittests or other kinds of assertions . If you
want extreme decisions to write maintainable assertions then use
nothing but True or False as the outcome, and even in this case you'll
be writing readable tests that could be reused for other purposes .
Abuse of string matching is probably wrong , e.g.

{{{
#!python

>>> unicode('aaa')
u'aaa'
}}}

... rather than

{{{
#!python

>>> x = unicode('aaa')
>>> type(x) is unicode # or x.__class__ I really don't use Py3k ;o)
True
>>> isinstance(x, unicode)
True
}}}

... unless you are really testing string matching (as could be the
case AFAICR of the issue about Zope test runner). Breaking things in
non-backwards compatible ways is worst, IMHO (especially in this case,
considering `doctest` limitations due to the fact that tests look like
interactive sessions by design ;o)

FWIW, besides, using `dutest` you have access  to the underlying test
case that wraps the particular interactive example and you can access
any of its methods. AFAICR looks like this

{{{
#!python

>>> __tester__.assertEquals(1, 100 / 100)
}}}

... and if `unittest` assertions are provided as standalone functions
then there's really no difference at all between both styles ;o).

> I think that's a
> terrible idea.

... to rely on string matching using non-standard features ? IMO yes,
but the same happens when writing TestCase(s) to perform string
matching . So I think that's not an drawback of the framework .


Therefore, to *ALWAYS* use doctests ? IMHO definitely no.

> I think it can work well in some situations, but it's
> not a tool to use to the exclusion of all others.
>

So I use doctest to write my tests 'cause

  - They are quite readable
  - ... so even users can read them and use that as a
     reference, tutorial, ...
  - ... and they can be part of the testing efforts and
     write doctests for you (e.g. user find a bug, you invite him
     to write a snippet illustrating what he was doing
     -rather than writing a unittest test case or alike-, and voilà !
     there you have a new test scenario ;o)
  - Doctests can be reused for many other SE-related tasks
  - I don't have time to write both docs and tests
    (especially when doing the same thing ;o)
  - ...

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
On adding Hessian (RPC) support for Trac -
http://feedproxy.google.com/~r/simelo-en/~3/Vit6dRudChU/on-adding-hessian-rpc-support-for-trac.html