[TIP] Use unittests for profiling?
Paul.Hildebrandt at disneyanimation.com
Fri May 14 09:34:07 PDT 2010
jason kirtland wrote:
> On Thu, May 13, 2010 at 7:24 AM, Matthew Wilson <matt at tplus1.com> wrote:
>> I've been writing lots of snippets of code and then feeding them into
>> cProfile and timeit, looking for stuff to optimize. It dawned on me
>> recently that these snippets look a lot like some of my unittest
>> classes: each snippet has a few statements and an optional setup
>> block. In both cases, I'm isolating a component from a bigger system
>> and then running just that component.
>> Has anyone done any work to reuse unit tests for profiling and timing?
>> Is there something theoretically wrong with this idea?
>> There's a practical hurdle -- the cProfile and timeit modules were
>> originally designed to eat strings of code. They can work with
>> callable objects, but I am finding that to be a little difficult when
>> I have non-trivial setups.
> The SQLAlchemy test suite implements a profiling test decorator that
> asserts that a test case uses only a certain number of function calls
> to get the job done.
Jason, why does it care? I know that there is some overhead in
function calls but is it really that large? I ask not as a criticism, I
am thinking about things like Matt is and want to know if I should
consider function calls aside from general timing.
> As written, it does not provide a direct measure
> of performance, more of a warning flag that goes up if a refactoring
> unexpectedly adds or removes overhead to a measured critical path.
> The general approach definitely can work, and with clever inspection
> of the stats you could do some interesting assertions on what happened
> during the run, or at least some tailored reporting if you're not
> interested in pass/fail semantics.
> I think we did have some timeit-based testing going on, but those were
> converted into non-tests because they were a bit slow to include in
> the regular full suite runs. I wasn't able to figure out how to
> account for the many system factors that affect real execution time
> and convert that into a definite pass or fail anyhow.
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
More information about the testing-in-python