[TIP] Bug with nosetests? Adding a new case with autospec=True will take 15 seconds

Sat Mar 9 16:34:04 PST 2013

Hi,
Sorry for the spam. Actually, I looked at it again. I feel like the way I
was patching was fundamentally incorrect.

#bitbucket/apis.py
import requests

def foo(..):
    requests.get(...)

then in my test I should just patch.object(requests, 'get') and not
patch('bitbucket.apis.requests', autospec=True)

I think that's one mistake I made, if I read the doc correctly.
Thanks.

John
On Sat, Mar 9, 2013 at 7:12 PM, John Wong <gokoproject at gmail.com> wrote:

> Thanks! Finally can get back to this.
> Yes. I ahve to agree that I should just patch out certain things like
> requests.get, and requests.post only.
>
> (bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$
> nosetests test_bitbucket.py
> ...
> ----------------------------------------------------------------------
> Ran 3 tests in 0.103s
>
> OK
>
>
> This is an individual test. It ran 3 tests. One of which is a new test
> class I tried to add (which just has one test case)
>
> If I comment out the new test class in test_bitbucket.py, and ran the
> entire suite:
> (bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$
> nosetests
> .................
> ----------------------------------------------------------------------
> Ran 17 tests in 1.020s
>
> OK
>
> As you can see. For some strange reasons, a new test class would trigger a
> big jump!
>
> I did a little debugging using nose's set_tace()
>
> run "nosetest" with debug: http://pastebin.com/YK7yXrtG
> run "nsoetest test_bitbucket.py" http://pastebin.com/WbfEYKnL
>
> I can understand the one with all tests would do some discovery so the
> debug log looks longer.
>
> John
>
> On Fri, Mar 8, 2013 at 8:40 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:
>
>>
>> On 7 Mar 2013, at 19:52, Dan Wandschneider <
>> daniel.wandschneider at schrodinger.com> wrote:
>>
>> > I have some guesses about this, which I elaborated on in:
>> > http://stackoverflow.com/a/15280350/1309332
>> >
>> > But basically, it looks like the recursive calls to create_autospec()
>> are slowing down on your mocked object (whatever it is).  ID recommend one
>> of three approaches:
>> > 1. Don't autospec.  Do you need it?
>> > 2. Only autospec some important method within the complex object that
>> you are replacing
>> > 3. use instance=True to limit the depth of recursion of calls to
>> create_autospec().  This may change behavior that you are depending upon,
>> however.  Be careful, since you are essentially lying to mock.patch if you
>> do this.
>> >
>>
>>
>> The issue is that adding one test (class/module ?)  slows the test run
>> down by 13 seconds or so. What is needed is running that test on its own
>> (with create_autospec on) to see if it actually takes that long or whether
>> there is some additional mysterious source of slowdown (which the OP seems
>> to suspect).
>>
>> Adding "instance=True" to *the patch call* is probably a bad idea.
>>
>> It is entirely *possible* that autospec is really, really slow in some
>> cases - but to be that much slower is very odd.
>>
>> Michael
>>
>> > Good luck!
>> > -Dan W.
>> >
>> >
>> >
>> > On Wed, Mar 6, 2013 at 12:00 PM, <
>> testing-in-python-request at lists.idyll.org> wrote:
>> > Send testing-in-python mailing list submissions to
>> >         testing-in-python at lists.idyll.org
>> >
>> > To subscribe or unsubscribe via the World Wide Web, visit
>> >         http://lists.idyll.org/listinfo/testing-in-python
>> > or, via email, send a message with subject or body 'help' to
>> >         testing-in-python-request at lists.idyll.org
>> >
>> > You can reach the person managing the list at
>> >         testing-in-python-owner at lists.idyll.org
>> >
>> > When replying, please edit your Subject line so it is more specific
>> > than "Re: Contents of testing-in-python digest..."
>> >
>> > Today's Topics:
>> >
>> >    1. Bug with nosetests? Adding a new case with autospec=True will
>> >       take 15 seconds (John Wong)
>> >
>> >
>> > ---------- Forwarded message ----------
>> > From: John Wong <gokoproject at gmail.com>
>> > To: testing-in-python at lists.idyll.org
>> > Cc:
>> > Date: Tue, 5 Mar 2013 22:57:29 -0500
>> > Subject: [TIP] Bug with nosetests? Adding a new case with autospec=True
>> will take 15 seconds
>> > HI,
>> >
>> > Thanks for all the help so far.
>> >
>> > Here is the repo:
>> https://bitbucket.org/yeukhon/bitbucket-python-api/src/6e57e95d5d09/tests/small?at=default
>> >
>> > I have a base unit test class in base.py.  For test_bitbucket.py, I
>> inherit from the base unit test class.
>> > I want to add new test cases and classes to test_bitbucket.py While
>> doing so, I discovered performance issue.
>> >
>> > I ran `nosetests` inside tests/small and I am getting these mix results
>> >
>> >
>> > Ran 18 tests in 14.523s        - autospec=True and
>> test_account_creation exists
>> >
>> > Ran 18 tests in 0.621s         - autospec=False and
>> test_account_creation_exists
>> >
>> >
>> > Ran 17 tests in 1.081s         - autospec=True and
>> test_account_creation is deleted
>> >
>> > Ran 17 tests in 0.090s         - autospec=False, and
>> test_account_creation is deleted
>> >
>> >
>> > I understand requests is a big library, but the performance hit is
>> crazy, going from 1.1s to 14.5s with and without the new test class,
>> respectively.
>> >
>> > Interestingly, if we run test_bitbucket.py individually.
>> >
>> > (bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$
>> nosetests test_bitbucket.py
>> > ...
>> > ----------------------------------------------------------------------
>> > Ran 3 tests in 0.090s
>> >
>> > OK
>> >
>> >
>> > Pretty much the same with/without autospec on.
>> >
>> > Any idea? I don't know enough about nosetests internal.
>> >
>> > Thanks!
>> >
>> > John
>> >
>> > _______________________________________________
>> > testing-in-python mailing list
>> > testing-in-python at lists.idyll.org
>> > http://lists.idyll.org/listinfo/testing-in-python
>> >
>> >
>> > _______________________________________________
>> > testing-in-python mailing list
>> > testing-in-python at lists.idyll.org
>> > http://lists.idyll.org/listinfo/testing-in-python
>>
>>
>> --
>> http://www.voidspace.org.uk/
>>
>>
>> May you do good and not evil
>> May you find forgiveness for yourself and forgive others
>> May you share freely, never taking more than you give.
>> -- the sqlite blessing
>> http://www.sqlite.org/different.html
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20130309/de64525b/attachment-0001.htm>