[TIP] Using coverage.py with code that uses sys.settrace()

[Leahy, Oliver]

Yes, I did notice that message, and it was helpful in identifying the problem.

From: bcannon at gmail.com [mailto:bcannon at gmail.com] On Behalf Of Brett Cannon
Sent: 30 December 2011 19:13
To: Ned Batchelder
Cc: Leahy, Oliver; testing-in-python at lists.idyll.org
Subject: Re: [TIP] Using coverage.py with code that uses sys.settrace()

Python's stdlib uses context managers to store the trace function before it gets modified and then sets it back later when the test exits; never even attempted to make coverage work while using another tracer. But we also benefit from code in our custom test runner to notice when the trace function is not put back after a test is run.
On Thu, Dec 29, 2011 at 13:49, Ned Batchelder <ned at nedbatchelder.com<mailto:ned at nedbatchelder.com>> wrote:
Yeah, sys.settrace() could be more sophisticated.  However you are setting the trace function, I would think you could manage to set it back somehow.  But even if you could, you wouldn't be getting coverage measurement for the time your trace function was in effect, unless you also call my trace function from yours.  A mess all around.

--Ned.


On 12/29/2011 4:22 AM, Leahy, Oliver wrote:
Thanks for the suggestions, I’ve tried the –timid flag, hasn’t helped. And unfortunately
(for me) it looks like I can’t successfully reset the trace function with settrace() because of
the way my trace functions work; I raise exceptions when I reached specified points in the
code, so once the exception is raised I don’t get an opportunity to call sys.settrace().

So I’m going to have to use mocks, as you suggest. What I really want is for sys.settrace() to
be able to manage a stack of functions that would be called in sequence ☺

Thanks again,
Ollie

From: Ned Batchelder [mailto:ned at nedbatchelder.com]
Sent: 28 December 2011 21:44
To: Leahy, Oliver
Cc: testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>
Subject: Re: [TIP] Using coverage.py with code that uses sys.settrace()

On 12/28/2011 10:58 AM, Leahy, Oliver wrote:
What’s the recommended way to use coverage.py with code that uses sys.settrace()

I’m using settrace() to simulate error conditions at certain lines during testing, but
this interferes with the use coverage.py makes of settrace() and I’m getting incorrect
coverage metrics.


I’m hacking at my code to try to store an existing value for sys.gettrace() whenever
I call settrace(),  and then restoring the pre-existing trace function whenever I stop
tracing using my function, but it’s not working for me at the moment.

Is this a sensible approach to this problem?  Would anyone have any other suggestions?
I don't know if anyone has tried measuring code that uses settrace.  You should be able to use gettrace and settrace to save and restore the coverage trace function, but I won't be that surprised if it isn't working.  You should try using the --timid flag on "coverage run" to see if it helps.  It's specifically for environments that do unusual things with settrace().  Are you sure settrace() is the best way to accomplish your goals? Mocking can also inject errors into running code, and may be easier to work with in the long run.

In any case, if you need more help, post more details.

--Ned.

Thanks,
Ollie



_______________________________________________

testing-in-python mailing list

testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>

http://lists.idyll.org/listinfo/testing-in-python

_______________________________________________
testing-in-python mailing list
testing-in-python at lists.idyll.org<mailto:testing-in-python at lists.idyll.org>
http://lists.idyll.org/listinfo/testing-in-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120103/2c1de7f6/attachment.htm>