[TIP] Bug with nosetests? Adding a new case with autospec=True will take 15 seconds

[Michael Foord]

On 7 Mar 2013, at 19:52, Dan Wandschneider <daniel.wandschneider at schrodinger.com> wrote:

> I have some guesses about this, which I elaborated on in:
> http://stackoverflow.com/a/15280350/1309332
> 
> But basically, it looks like the recursive calls to create_autospec() are slowing down on your mocked object (whatever it is).  ID recommend one of three approaches:
> 1. Don't autospec.  Do you need it?
> 2. Only autospec some important method within the complex object that you are replacing
> 3. use instance=True to limit the depth of recursion of calls to create_autospec().  This may change behavior that you are depending upon, however.  Be careful, since you are essentially lying to mock.patch if you do this.
> 


The issue is that adding one test (class/module ?)  slows the test run down by 13 seconds or so. What is needed is running that test on its own (with create_autospec on) to see if it actually takes that long or whether there is some additional mysterious source of slowdown (which the OP seems to suspect).

Adding "instance=True" to *the patch call* is probably a bad idea.

It is entirely *possible* that autospec is really, really slow in some cases - but to be that much slower is very odd.  

Michael

> Good luck!
> -Dan W.
> 
> 
> 
> On Wed, Mar 6, 2013 at 12:00 PM, <testing-in-python-request at lists.idyll.org> wrote:
> Send testing-in-python mailing list submissions to
>         testing-in-python at lists.idyll.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://lists.idyll.org/listinfo/testing-in-python
> or, via email, send a message with subject or body 'help' to
>         testing-in-python-request at lists.idyll.org
> 
> You can reach the person managing the list at
>         testing-in-python-owner at lists.idyll.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
> 
> Today's Topics:
> 
>    1. Bug with nosetests? Adding a new case with autospec=True will
>       take 15 seconds (John Wong)
> 
> 
> ---------- Forwarded message ----------
> From: John Wong <gokoproject at gmail.com>
> To: testing-in-python at lists.idyll.org
> Cc: 
> Date: Tue, 5 Mar 2013 22:57:29 -0500
> Subject: [TIP] Bug with nosetests? Adding a new case with autospec=True will take 15 seconds
> HI,
> 
> Thanks for all the help so far.
> 
> Here is the repo: https://bitbucket.org/yeukhon/bitbucket-python-api/src/6e57e95d5d09/tests/small?at=default
> 
> I have a base unit test class in base.py.  For test_bitbucket.py, I inherit from the base unit test class.
> I want to add new test cases and classes to test_bitbucket.py While doing so, I discovered performance issue.
> 
> I ran `nosetests` inside tests/small and I am getting these mix results
> 
> 
> Ran 18 tests in 14.523s        - autospec=True and test_account_creation exists
> 
> Ran 18 tests in 0.621s         - autospec=False and test_account_creation_exists
> 
> 
> Ran 17 tests in 1.081s         - autospec=True and test_account_creation is deleted
> 
> Ran 17 tests in 0.090s         - autospec=False, and test_account_creation is deleted
> 
> 
> I understand requests is a big library, but the performance hit is crazy, going from 1.1s to 14.5s with and without the new test class, respectively.
> 
> Interestingly, if we run test_bitbucket.py individually.
> 
> (bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$ nosetests test_bitbucket.py
> ...
> ----------------------------------------------------------------------
> Ran 3 tests in 0.090s
> 
> OK
> 
> 
> Pretty much the same with/without autospec on.
> 
> Any idea? I don't know enough about nosetests internal.
> 
> Thanks!
> 
> John
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
> 
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html