[TIP] Python Testing book review

Tarek Ziadé ziade.tarek at gmail.com
Sun Mar 7 07:56:02 PST 2010

On Sun, Mar 7, 2010 at 3:10 PM, Jim Fulton <jim at zope.com> wrote:
> On Sat, Mar 6, 2010 at 2:25 PM, Tarek Ziadé <ziade.tarek at gmail.com> wrote:
> ...
>> I've done that with my latest book for most examples, but you cannot
>> have a fully-covered book because some code snippets cannot be
>> doctests. You would need to write test fixtures that would be longer
>> than the book itself to have a good test coverage because a book is
>> simply not an application.
> Can you give one or two examples of what you're thinking of when you
> say "some code snippets cannot be doctests"?

For example, if you want to present how a class is structured and want
to show a layout example,
it's not a class that is used in any example in the book (unlike in a
doctest that goes with an application or a library)

So you have some explanation text, and besides, a class example:

This is how you write a class :

    class Foo(object):
        """your doc test"""
        def method(self):
            # your code goes here
            print 'Foo.method


Simple. Yet you could do a typo in this snippet. I have forgot to
close the string here.
And the reader that reproduces the snippet will be unhappy with that.


Using doctest would imply to have a python prompt, with a specific
layout on ">>>" and "..."
*and* to test the class method in an example I guess. For example:

This is how you write a class :

    >>> class Foo(object):
    ...    """your doc test"""
    ...     def method(self):
    ...        # your code goes here
    ...        print 'Foo.method
    >>> foo = Foo()
    >>> foo.method()

Here, my text is turned into a doctest, but that is something I don't
want because it turns my text + example into a heavy structure. And I
simply don't want here in my book to have a full example.

If I do it nevertheless, to avoid typo, I could maybe hide my test
into some script, so the final output looks like this in the book:

This is how you write a class :

    >>> class Foo(object):
    ...    """your doc test"""
    ...     def method(self):
    ...        # your code goes here
    ...        print 'Foo.method

Then I would have a script that tries the code snippet by importing
the Foo class:

    def test_chapter1():
        from chapter1_doctest import Foo
        foo = Foo()
        # asserting that this method prints 'Foo.method' here
        # so we catch the typo

*or* maybe I could simply work on loading the examples in the Python
parser ala Pyflakes,
that makes sure there are no syntax error.

(I realize now that looking for syntax error is very simple, and that
80% of my typos are syntax errors)

But I can still have some problems with grammar-compliant code, since
it's not executed unless there's a test.

So at the end, for some code snippets, this requires a *lot* of extra
work that probably
don't worth the effort I think. The time taken by a reviewer to try
and review the snippet is < the time taken to write the test fixture +

> Did you try using manuel, http://packages.python.org/manuel/?  I've
> found many code examples that don't fit the doctest python-prompt
> style, either because they are module source, or because they aren't
> even Python. I'm very happy with how the bobo dcumentation,
> http://bobo.digicool.com/, turned out with manuel's help. Almost all
> of the bobo examples are tested even though none are in the
> traditional doctest style.

No, I wasn't aware of that project, it looks like it fixes some issues
I have mentioned. I'll have a try!

If it doesn't do it already, (I didn't read the whole doc) it could
try to look for syntax errors in code snippets even if they are not
tested, as I mentioned. That would cut down typos in doc imho.


Tarek Ziadé | http://ziade.org

More information about the testing-in-python mailing list