[TIP] Docstrings in Test methods

[Mike Pirnat]

On Sat, Nov 5, 2011 at 9:07 AM, Alfredo Deza <alfredodeza at gmail.com> wrote:
> But this would be for failing tests you say? This is what I would see
> in Python2.6
> and unittest 1.63
>
> $ python test_foo.py
> F
> ======================================================================
> FAIL: when foo uses bar as a connection it should be True when successful
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>  File "test_foo.py", line 7, in test_foo_should_be_true
>    assert False
> AssertionError
>
> ----------------------------------------------------------------------
> Ran 1 test in 0.000s
>
> The test method *does* appear. Is this *not* the case in your code base?

In a trivial case, this would be okay.  We have ~1M LoC, ~10k files,
and a stack trace that's usually ~10 lines or more, and the people who
write docstrings for tests tend not to be that succinct, so you end up
with (essentially) garbage in the "FAIL" or "ERROR" line, and then you
get to go sifting through the stack trace to see exactly which test is
hurting.

I really prefer to see:

FAIL: package.subpackage.tests.test_module.TestClass.test_which_failed

Instead of:

FAIL: Test obscure frobulator that no one has ever heard of; lengthy
discussion about the state of modern frobulation blah blah blah this
is why we can't have nice things

The former lets me quickly see where in the codebase the failing test
is, and if I want to learn more about the test it's just a quick "vim
<troublemaker>" away.  Though I often just want to redirect blame to
the appropriate place. ;-)

I find that awkward test method names are a sign that I need to
rethink what I'm trying to express in the test.

-- 
Mike Pirnat
mpirnat at gmail.com
http://www.pirnat.com/