[TIP] Docstrings in Test methods

Ben Finney ben+python at benfinney.id.au
Fri Nov 25 11:15:42 PST 2011

I have been asked for my opinion on this thread that I missed, so I'll
respond to a few salient messages.

Mike Pirnat <mpirnat at gmail.com> writes:

> On Sat, Nov 5, 2011 at 9:07 AM, Alfredo Deza <alfredodeza at gmail.com> wrote:
> > The test method *does* appear. Is this *not* the case in your code base?
> In a trivial case, this would be okay.  We have ~1M LoC, ~10k files,
> and a stack trace that's usually ~10 lines or more, and the people who
> write docstrings for tests tend not to be that succinct

That argues for better docstrings, and seeing the bad docstrings should
be a spur to writing better ones.

> I really prefer to see:
> FAIL: package.subpackage.tests.test_module.TestClass.test_which_failed

I'd like to see that too.

> Instead of:

Why “instead of”?

> FAIL: Test obscure frobulator that no one has ever heard of; lengthy
> discussion about the state of modern frobulation blah blah blah this
> is why we can't have nice things

That's why I conform to PEP 257 for docstrings, and docstrings for test
cases are no different. Your dislike of the badly-written docstrings in
your code base are another indication that better (i.e. PEP 257
compliant) docstrings are called for.

That way, the one-line synopsis is what can be displayed in the test
report output. The longer descriptive remainder of the docstring (if it
exists; most docstrings don't need to be longer than a one-line
synopsis) isn't used in the report.

> I find that awkward test method names are a sign that I need to
> rethink what I'm trying to express in the test.

Agreed, the test case name needs to be descriptive. But the docstring
synopsis can often express it in human language more readably, and
that's important in a report full of output.

So both are needed: good test case names, and good test case docstrings.

Michael Foord <michael at voidspace.org.uk> writes:

> Output from unittest2 (and Python 2.7):
> $ unit2 discover -v
> test_something (test_thing.SomeTest)
> This test has a docstring ... ERROR
> ======================================================================
> ERROR: test_something (test_thing.SomeTest)
> This test has a docstring
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "/compile/temp/test_thing.py", line 7, in test_something
>     1/0
> ZeroDivisionError: integer division or modulo by zero
> ----------------------------------------------------------------------
> Ran 1 test in 0.022s
> The docstring is still output in verbose mode - but *after* the test
> name.

Great! So long as this uses the synopsis of the docstring (and not the
full docstring), it looks good for my purposes.

Gary Bernhardt <gary.bernhardt at gmail.com> writes:

> Exactly: the test class's name refers to the state of the world that
> the test expects. That world is created in the setup. These things
> aren't in opposition; they're the same!

Right. Naming the test case class provides the state-of-the-world
context, and the test case name and its docstring synopsis then have
plenty of room to describe the “when I do this, that should happen”
assertion of the specific test case.

Jonathan Lange <jml at mumak.net> writes:

> That's because many test runners use TestCase.shortDescription(),
> rather than TestCase.id(). I personally think that shortDescription is
> bogus for the reasons you suggest

I hope you can see why that's not so: shortDescription is very useful
because it makes use of the docstring for valuable information about the
test case.

> so the runners I've had a hand in all use id().

Feel free to use that too, but don't ignore my docstrings :-)

 \     “I got some new underwear the other day. Well, new to me.” —Emo |
  `\                                                           Philips |
_o__)                                                                  |
Ben Finney

More information about the testing-in-python mailing list