[TIP] Bug with nosetests? Adding a new case with autospec=True will take 15 seconds

Sat Mar 9 16:12:52 PST 2013

Thanks! Finally can get back to this.
Yes. I ahve to agree that I should just patch out certain things like
requests.get, and requests.post only.

(bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$
nosetests test_bitbucket.py
...
----------------------------------------------------------------------
Ran 3 tests in 0.103s

OK

This is an individual test. It ran 3 tests. One of which is a new test
class I tried to add (which just has one test case)

If I comment out the new test class in test_bitbucket.py, and ran the
entire suite:
(bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$
nosetests
.................
----------------------------------------------------------------------
Ran 17 tests in 1.020s

OK

As you can see. For some strange reasons, a new test class would trigger a
big jump!

I did a little debugging using nose's set_tace()

run "nosetest" with debug: http://pastebin.com/YK7yXrtG
run "nsoetest test_bitbucket.py" http://pastebin.com/WbfEYKnL

I can understand the one with all tests would do some discovery so the
debug log looks longer.

John

On Fri, Mar 8, 2013 at 8:40 AM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>
> On 7 Mar 2013, at 19:52, Dan Wandschneider <
> daniel.wandschneider at schrodinger.com> wrote:
>
> > I have some guesses about this, which I elaborated on in:
> > http://stackoverflow.com/a/15280350/1309332
> >
> > But basically, it looks like the recursive calls to create_autospec()
> are slowing down on your mocked object (whatever it is).  ID recommend one
> of three approaches:
> > 1. Don't autospec.  Do you need it?
> > 2. Only autospec some important method within the complex object that
> you are replacing
> > 3. use instance=True to limit the depth of recursion of calls to
> create_autospec().  This may change behavior that you are depending upon,
> however.  Be careful, since you are essentially lying to mock.patch if you
> do this.
> >
>
>
> The issue is that adding one test (class/module ?)  slows the test run
> down by 13 seconds or so. What is needed is running that test on its own
> (with create_autospec on) to see if it actually takes that long or whether
> there is some additional mysterious source of slowdown (which the OP seems
> to suspect).
>
> Adding "instance=True" to *the patch call* is probably a bad idea.
>
> It is entirely *possible* that autospec is really, really slow in some
> cases - but to be that much slower is very odd.
>
> Michael
>
> > Good luck!
> > -Dan W.
> >
> >
> >
> > On Wed, Mar 6, 2013 at 12:00 PM, <
> testing-in-python-request at lists.idyll.org> wrote:
> > Send testing-in-python mailing list submissions to
> >         testing-in-python at lists.idyll.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> >         http://lists.idyll.org/listinfo/testing-in-python
> > or, via email, send a message with subject or body 'help' to
> >         testing-in-python-request at lists.idyll.org
> >
> > You can reach the person managing the list at
> >         testing-in-python-owner at lists.idyll.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of testing-in-python digest..."
> >
> > Today's Topics:
> >
> >    1. Bug with nosetests? Adding a new case with autospec=True will
> >       take 15 seconds (John Wong)
> >
> >
> > ---------- Forwarded message ----------
> > From: John Wong <gokoproject at gmail.com>
> > To: testing-in-python at lists.idyll.org
> > Cc:
> > Date: Tue, 5 Mar 2013 22:57:29 -0500
> > Subject: [TIP] Bug with nosetests? Adding a new case with autospec=True
> will take 15 seconds
> > HI,
> >
> > Thanks for all the help so far.
> >
> > Here is the repo:
> https://bitbucket.org/yeukhon/bitbucket-python-api/src/6e57e95d5d09/tests/small?at=default
> >
> > I have a base unit test class in base.py.  For test_bitbucket.py, I
> inherit from the base unit test class.
> > I want to add new test cases and classes to test_bitbucket.py While
> doing so, I discovered performance issue.
> >
> > I ran `nosetests` inside tests/small and I am getting these mix results
> >
> >
> > Ran 18 tests in 14.523s        - autospec=True and test_account_creation
> exists
> >
> > Ran 18 tests in 0.621s         - autospec=False and
> test_account_creation_exists
> >
> >
> > Ran 17 tests in 1.081s         - autospec=True and test_account_creation
> is deleted
> >
> > Ran 17 tests in 0.090s         - autospec=False, and
> test_account_creation is deleted
> >
> >
> > I understand requests is a big library, but the performance hit is
> crazy, going from 1.1s to 14.5s with and without the new test class,
> respectively.
> >
> > Interestingly, if we run test_bitbucket.py individually.
> >
> > (bbpy)yeukhon at yeukhon-P5E-VM-DO:~/hg/bitbucket-python-api/tests/small$
> nosetests test_bitbucket.py
> > ...
> > ----------------------------------------------------------------------
> > Ran 3 tests in 0.090s
> >
> > OK
> >
> >
> > Pretty much the same with/without autospec on.
> >
> > Any idea? I don't know enough about nosetests internal.
> >
> > Thanks!
> >
> > John
> >
> > _______________________________________________
> > testing-in-python mailing list
> > testing-in-python at lists.idyll.org
> > http://lists.idyll.org/listinfo/testing-in-python
> >
> >
> > _______________________________________________
> > testing-in-python mailing list
> > testing-in-python at lists.idyll.org
> > http://lists.idyll.org/listinfo/testing-in-python
>
>
> --
> http://www.voidspace.org.uk/
>
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing
> http://www.sqlite.org/different.html
>
>
>
>
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20130309/8f2d4816/attachment.htm>