[TIP] Unittest Changes

[Michael Foord]

Jonathan Lange wrote:
> On Mon, Jul 21, 2008 at 7:16 AM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
>   
>> Unless someone beats me to it I'm intending to work on some improvements
>> to unittest after I have finished "IronPython in Action", which
>> realistically will be a couple more months.
>>
>>     
>
> Hi Michael,
>
> I've already started a project along these lines, temporarily called pyunit3k:
>     http://launchpad.net/pyunit3k
>
> I haven't announced it until now because the name sucks and because
> it's test coverage is still rather poor. I would love it if you could
> take a look at it and tell me what you think.
>
> The aim is to incorporate improvements to unittest that have stood the
> test of time in real projects, rather than to come up with blue-sky
> features.
>   

Right - although virtually every project I've worked on with unittest 
had ended up with an ad-hoc test collection mechanism, so I think the 
'discover_tests' features I've discussed are prima-facie needed.

It looks like Twisted and Bzr both have prior-art in these areas.

Great to see some movement, and when I get time free to work on this 
I'll have a look at what you've done with pyunit3k. Are you targeting 
Python 2.7 as well?

I've never had a need for per-module / per-class cleanup methods myself. 
I tend to prefer creating needed resources by decorating test-methods, 
even if this means creating them 'per-test'. As such I'm not so 
convinced about 'addCleanup' etc.

In the end if these changes are to go back into unittest (which is my 
goal) then it will have to go through a PEP anyway.

Michael





>   
>> The biggest way I can think of improving unittest is to add automatic
>> test discovery:
>>
>>    unittests.discover_tests(path='.', pattern='test*.py', recurse=True)
>>
>> This will simply search the path provided (defaulting to the current
>> directory) matching test files based on the pattern.
>>
>>     
>
> At the risk of laboring a point, twisted.trial does this already —
> pinch the code from there (or from nose, I guess). In trial's case,
> the tests are already written.
>
>   
>> Another change will be the improvement of failure message for
>> assertEquals. When comparing long strings it will show a diff, when
>> comparing containers it will show you the members that are different
>> (lists, sets, dictionaries).
>>
>>     
>
> bzrlib has some code along these lines.
>
> I can't recall if pyunit3k has pinched it.
>
>   
>> Other minor changes - have assertRaises return the exception instance so
>> that you can assertions about it, possibly container membership
>> assertion methods, nothing spectacular.
>>
>>     
>
> pyunit3k has this.
>
>   
>> Any other good ideas about ways that unittest could change that don't
>> either grow the API greatly or break backwards compatibility?
>>
>>     
>
> The big one is addCleanup (again, implemented & used in pyunit3k,
> Twisted and bzrlib). The idea is that it allows you to dynamically
> create resources in tests without worry.
>
> Here's an example of usage:
>
>     def makeTemporaryDirectory(self):
>         d = tempfile.mkdtemp()
>         self.addCleanup(shutil.rmtree, d)
>         return d
>
> Here's a rough implementation:
>
> def addCleanup(self, f, *a, **kw):
>     self._cleanup_stack.append((f, *a, **kw))
>
> def _runCleanups(self, test_result):
>     for f, *a, **kw in self._cleanup_stack:
>         try:
>             f(*a, **kw)
>         except:
>             test_result.addError(self, sys.exc_info())
>
>
> _runCleanups is *always* called, even if setUp() fails (pyunit doesn't
> run tearDown if setUp fails).
>
> Once you start writing tests with this, it's very hard to go back.
>
> Thank you for taking the initiative to work at improving unittest.
>
> jml
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.resolverhacks.net/
http://www.theotherdelia.co.uk/