[TIP] Fwd: [Launchpad-dev] Reminder about a great Testtools matcher

[Victoria G. Laidler]

Herman Sheremetyev wrote:
> On Wed, Jul 20, 2011 at 10:08 AM, Martin Pool <mbp at sourcefrog.net> wrote:
>   
>> I thought this was cool enough it might be of interest to t-i-p.
>> Martin
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Julian Edwards <julian.edwards at canonical.com>
>> Date: 19 July 2011 22:11
>> Subject: [Launchpad-dev] Reminder about a great Testtools matcher
>> To: Launchpad Community Development Team <launchpad-dev at lists.launchpad.net>
>>
>>
>> I hope you all have the documents for Testtools[1] matchers on speed dial
>> because they are awesome.  I thought I'd remind you about the
>> MatchesStructure
>> matcher because I used it a while ago, forgot about it, and re-discovered it
>> recently.
>>
>> Check out this diff snippet:
>>
>> -        self.assertEqual(source_name, copy_job.package_name)
>> -        self.assertEqual(version, copy_job.package_version)
>> -        self.assertEqual(target_archive, copy_job.target_archive)
>> -        self.assertEqual(source_archive, copy_job.source_archive)
>> -        self.assertEqual(to_series, copy_job.target_distroseries)
>> -        self.assertEqual(to_pocket, copy_job.target_pocket)
>> -        self.assertFalse(copy_job.include_binaries)
>> -        self.assertEquals(PackageCopyPolicy.INSECURE, copy_job.copy_policy)
>> +        self.assertThat(copy_job, MatchesStructure(
>> +            package_name=Equals(source_name),
>> +            package_version=Equals(version),
>> +            target_archive=Equals(target_archive),
>> +            source_archive=Equals(source_archive),
>> +            target_distroseries=Equals(to_series),
>> +            target_pocket=Equals(to_pocket),
>> +            include_binaries=Equals(False),
>> +            copy_policy=Equals(PackageCopyPolicy.INSECURE)))
>>
>> I replaced a load of serially-processed assertions with a single assertion.
>> This is good because:
>>
>>  * It checks all of the conditions, not just up until the first one fails.
>>  * It prints out all the values if any of them fail to match.
>>
>> I hope to see more use of this pattern!
>>     
>
> I hope not. I would hope to see more smaller tests rather than armies
> of asserts in a single test. Yes, it's necessary to have multiple
> asserts per test at times, but in a case like the above where you're
> comparing properties on the same object, the fact that you're doing
> separate asserts should steer you toward writing a better equality
> operator on that object rather than bunching a ton of asserts into a
> single statement.
>   
I also tend towards "smaller tests rather than armies of asserts", but 
note that the case as presented identifies a situation where the desired 
test granularity is coarse (either pass or fail), but the desired test 
*reporting* is fine (tell me _what_ passed or failed).

This is a common real-world scenario (at least in my real world!) and 
it's the kind of thing we wrote Pandokia[1] to handle. The test author 
can write a coarse test with fine-grained reporting by means of test 
definition attributes and test result attributes (TDAs and TRAs) that 
are set in the test and can be reported out in a key-value format that 
Pandokia can read, parse, and store in a database to make nice reports. 
The attributes are not limited to pass/fail but can include computed 
values, differences, anything that might be useful in figuring out why a 
test failed, or by how much it exceeded its pass/fail threshold.

cheers,
Vicki Laidler

[1] - https://svn.stsci.edu/trac/ssb/etal/wiki/WikiStart#pandokia