[TIP] Docstrings in Test methods

[Alfredo Deza]

On Sat, Nov 5, 2011 at 10:20 AM, Mike Pirnat <mpirnat at gmail.com> wrote:
> On Sat, Nov 5, 2011 at 9:07 AM, Alfredo Deza <alfredodeza at gmail.com> wrote:
>> But this would be for failing tests you say? This is what I would see
>> in Python2.6
>> and unittest 1.63
>>
>> $ python test_foo.py
>> F
>> ======================================================================
>> FAIL: when foo uses bar as a connection it should be True when successful
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>  File "test_foo.py", line 7, in test_foo_should_be_true
>>    assert False
>> AssertionError
>>
>> ----------------------------------------------------------------------
>> Ran 1 test in 0.000s
>>
>> The test method *does* appear. Is this *not* the case in your code base?
>
> In a trivial case, this would be okay.  We have ~1M LoC, ~10k files,
> and a stack trace that's usually ~10 lines or more, and the people who
> write docstrings for tests tend not to be that succinct, so you end up
> with (essentially) garbage in the "FAIL" or "ERROR" line, and then you
> get to go sifting through the stack trace to see exactly which test is
> hurting.

Ok, so this could be also because you have bad docstrings? If all your
docstrings where
succinct, granular and descriptive enough, would you still think no
docstrings is better?
>
> I really prefer to see:
>
> FAIL: package.subpackage.tests.test_module.TestClass.test_which_failed
>
> Instead of:
>
> FAIL: Test obscure frobulator that no one has ever heard of; lengthy
> discussion about the state of modern frobulation blah blah blah this
> is why we can't have nice things
>
> The former lets me quickly see where in the codebase the failing test
> is, and if I want to learn more about the test it's just a quick "vim
> <troublemaker>" away.  Though I often just want to redirect blame to
> the appropriate place. ;-)
>
> I find that awkward test method names are a sign that I need to
> rethink what I'm trying to express in the test.
>
> --
> Mike Pirnat
> mpirnat at gmail.com
> http://www.pirnat.com/
>