[TIP] TiP BoF feedback & call for help

[Greg Turnquist]

As they say, premature optimization is the root of all evil. Profile and
determine where all your time is being spent.

On Mon, Mar 14, 2011 at 5:05 PM, Mark Sienkiewicz <sienkiew at stsci.edu>wrote:

>
>  So who is wrong? me? us? jenkins? django? unittest ? Who to blame ?
>> I'll love to run my tests more but 30 minutes feedback is just crazy.
>>
>>
>
> You'll just have to start measuring things.  Blaming django doesn't get you
> anywhere even if it is accurate -- you have to understand what is happening
> in your system so you can know what you can do different.
>
> First, how do you know there is anything wrong?  Your average throughput
> works out to about 1.8 seconds per test.  Is it possible that 20-30 minutes
> is actually _fast_ for what you are doing?  If not, how do you know?
>
> Are all the tests taking 1.8 seconds +/- 2% ?  Or do 90% of your tests take
> 0.1 seconds each and 10% of your tests take 17 seconds each?  What do the
> fast ones have in common?  What do the slow ones have in common?
>
> If you run a single test, how long does it take?  If you run 10 tests, does
> it take 10 times as long?  Do 100 tests take 100 times as long?
>
> If you're using unittest, how long does it spend in the test method, how
> long in the class setup, how long in the overhead code?
>
> Do your tests log in to the postgres server 500 times, or only once?  And
> are the postgres tests faster or slower than the same tests run on sqlite?
>
> Do you run all your tests in a single process?  If not, how many processes
> and how much process creation time?  (I ran truss on my python interpreter
> once and saw about 3000 file open calls just to start an interactive
> interpreter.)
>
> Once you collect some data (maybe even just a few minutes - you might not
> need a whole test run), you have some clues.  There is no point in
> optimizing until you know what needs to be better.  If you just poke about
> at random, you might completely optimize something that is responsible for
> 1% of your run time -- but you don't care if you save 18 seconds out of half
> an hour.
>
> Mark
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>



-- 
Greg Turnquist (Greg.L.Turnquist at gmail.com)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110314/d0b8b606/attachment.htm>