[TIP] Asserting behaviour of exception tracebacks

[Michael Foord]

Ben Finney wrote:
> Howdy all,
>
> I have some code that handles exceptions from a lower level, applies
> contextual knowledge where the exception is caught, and raises a
> more-informative error. The error is wrapped in a module specific
> error class::
>
>     class ModuleSpecificError(RuntimeError):
>         def __init__(self, orig, context):
>             self.orig = orig
>             self.context = context
>
>         def __str__(self):
>             orig_name = orig.__class__.__name__
>             msg = (
>                 "failed during %(context)s"
>                 " (%(orig_name)s: %(orig)s)"
>                 ) % vars(self)
>
> That's okay as far as it goes. (It even includes the original error
> classname and message in the ModuleSpecificError message.)
>
> An example of code that's currently using this might be::
>
>     def problem():
>         empty = {}
>         # This will raise a KeyError
>         foo = empty['bar']
>
>     def handle_problem():
>         # Some arbitrary context for the error
>         context = objecT()
>         try:
>             problem()
>         except Exception, exc:
>             error = ModuleSpecificError(orig=exc, context=context)
>             raise error
>
> But this makes the classic mistake of losing the traceback of the
> original error: the ModuleSpecificError gets raised with a traceback
> starting at the 'raise error' statement, which is useless for
> debugging.
>
> The solution for this is to change the handling to preserve the
> traceback::
>
>     def handle_problem():
>         # Some arbitrary context for the error
>         context = objecT()
>         try:
>             problem()
>         except Exception, exc:
>             exc_class, exc_instance, exc_traceback = sys.exc_info()
>             error = ModuleSpecificError(orig=exc, context=context)
>             raise error, None, exc_traceback
>
> That will raise the specific error instance I want, complete with its
> informative context-specific message; but the traceback will lead all
> the way to the original problem (the statement that causes a
> KeyError).
>
>
> Great, I now have an implementation change I can make to alter the
> application behaviour. The issue is: how can I test this change? I'm
> practicing TDD/BDD, and this is a change in externally-visible
> behaviour. I therefore need a unit test to assert that the behaviour
> has actually changed as specified.
>
> The 'unittest' module in Python gives me 'TestCase.failUnlessRaises'
> to assert that an error of a specific class *is* raised from a
> particular function call. I can't see a clean way to assert that
> *when* that exception is raised, the traceback contains the original
> error occurence and doesn't just stop at the ModuleSpecificError
> occurrence.
>
> I don't even know how I'd really test that in code; even if I did know
> how, I'd immediately want to be able to parameterise it so that I
> don't have to keep writing it over again. How can I go about doing
> that?
>
>   
One convenience method we have in our test framework is 
'assertRaisesWithMessage'. I don't have the code to hand, but it should 
be obvious what it does... :-)

In your case you need to ask 'why' you want to preserve the original 
traceback. The answer to that  question will tell you what you ought to 
be testing.

For example, if you want the information preserved for the traceback 
message, then you could test it something like this:

from traceback import format_exc
try:
    do_something()
except ExpectedException, e:
    pass
else:
    self.fail('ExpectedException was not raised')


traceback = format_exc()
self.assertSomethingAboutTraceback(traceback)

I wouldn't bother generalising until the second time you need it (when 
you really know which parts are general).

HTH

Michael
http://www.manning.com/foord