[TIP] Asserting behaviour of exception tracebacks

[Michael Foord]

Ben Finney wrote:
> [Please don't send me copies of messages posted to the list. I
> participate here via a non-mail interface, and it's annoying to get
> the messages appearing duplicated in email.]
>   

This list is setup so that in most mail clients it requires a 
'reply-all' to keep the reply on the list. I've manually removed your 
address this time but make no promises about the future.
> Michael Foord <fuzzyman at voidspace.org.uk> writes:
>
>   
>> Ben Finney wrote:
>>     
>>> The solution for this is to change the handling to preserve the
>>> traceback::
>>> [.....]
>>>
>>> That will raise the specific error instance I want, complete with
>>> its informative context-specific message; but the traceback will
>>> lead all the way to the original problem (the statement that
>>> causes a KeyError).
>>>
>>>
>>> Great, I now have an implementation change I can make to alter the
>>> application behaviour. The issue is: how can I test this change?
>>> I'm practicing TDD/BDD, and this is a change in externally-visible
>>> behaviour. I therefore need a unit test to assert that the
>>> behaviour has actually changed as specified.
>>>
>>>       
>> One convenience method we have in our test framework
>>     
>
> Which framework is that?
>   
Based on unittest.

>   
>> is 'assertRaisesWithMessage'. I don't have the code to hand, but it
>> should be obvious what it does... :-)
>>     
>
> Hmm, yes. I suppose I'll end up having my custom 'TestCase' grow a
> method for this eventually.
>
>   
>> In your case you need to ask 'why' you want to preserve the original
>> traceback. The answer to that question will tell you what you ought
>> to be testing.
>>     
>
> That's a fair question. I want to preserve the original traceback
> because otherwise I'm throwing away information useful for the person
> who will be debugging the exception.
>
>   
>> For example, if you want the information preserved for the traceback
>> message
>>     
>
> Not just for the traceback *message*, but for *any* purpose to which a
> traceback object can be put. Any custom exception handler should get
> the traceback object leading all the way to the original exception
> that was raised.
>
>   
>> then you could test it something like this:
>>
>> from traceback import format_exc
>> try:
>>     do_something()
>> except ExpectedException, e:
>>     pass
>> else:
>>     self.fail('ExpectedException was not raised')
>>
>> traceback = format_exc()
>> self.assertSomethingAboutTraceback(traceback)
>>     
>
> In the tests I have in mind, the environment is rigged by the unit
> test to raise an exception object created specifically for the
> purpose. So, I have that exception object available to make assertions
> about; I don't need to interpose the abstraction of 'format_exc'.
>
> What I don't know how to do is make a simple assertion about a
> traceback object to find the exception that originally caused it, and
> compare it against the original exception object that the unit test
> rigged to happen.
>   


Well - if you find a *use-case* for the traceback object - then you can 
test against that use case. All the information is there  - including in 
the formatted text of the traceback, so you could just assert that some 
information from the original exception is *in* that text.

Michael