[TIP] Test case docstrings are valuable and useful in test reports (was: nose_fixes 1.1 released!)

[Ben Finney]

Jonathan Lange <jml at mumak.net> writes:

> I certainly find comments/docstrings on tests to be highly useful[1],
> but generally I get all of the utility when I see a failing test, go
> look it up in my editor and then read the docstring.

I see utility in seeing the test description along with *whatever* the
test result is. It's not only when a test is failing.

A simple example: the test is one of several that are affected by the
code I just wrote. I want to see human-readable test descriptions for
all the tests, along with the results, for proper context of what's
working and what's not.

> I'm keen to learn if there's a better way though. What sort of
> docstrings do you write? Do you like the first line of the docstring
> displayed, or the whole thing?

Described in my earlier message:

> >> On Thu, Nov 24, 2011 at 1:18 AM, Ben Finney <ben+python at benfinney.id.au> wrote:
> >
> >> > When I write a docstring for a test, it is in the expectation
> >> > that test reports will use the synopsis (the first line of the
> >> > PEP 257 compliant docstring) as the test description.


Jonathan Lange <jml at mumak.net> writes:

> How does the content of the docstring illuminate where the name of the
> test doesn't?

The ‘id’ of the test function is usually fully-qualified, which means
“really long” since a well-written test function will have a descriptive
name.

So the test description will say what the test is for in human language,
without all the elaborate hierarchical qualifiers. It has a full
docstring synopsis in which to say it, without loads of characters
prefixed to it.

And yes, I'll phrase the synopsis as an admonition: “Foo should do Bar
under Baz conditions.”.

> Some test runners show tests twice: once while running them and once
> again in a summary if they fail: does it help to have the docstring in
> both places?

Yes, it helps.

> Sorry to give you the third degree, but you're the first person I know
> who likes this behaviour, and I want to know more.

Glad to have the opportunity to explain.

-- 
 \            “If you continue running Windows, your system may become |
  `\        unstable.” —Microsoft, Windows 95 bluescreen error message |
_o__)                                                                  |
Ben Finney