<br><br><div class="gmail_quote">On Mon, Feb 14, 2011 at 8:12 PM, Herman Sheremetyev <span dir="ltr"><<a href="mailto:herman@swebpage.com">herman@swebpage.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hey everyone,<br>
<br>
I've seen a lot of messages on this list of people mentioning they use<br>
py.test, so I'm curious what advantages it offers. I gave it a try and<br>
immediately noticed it was literally 10x slower than nose, taking 0.60<br>
seconds to run the same set of tests it takes nose 0.062. As a result<br>
I pretty much wrote it off since test speed is pretty important to me,<br>
and all the red in my terminal seemed a bit unnecessary. But as I see<br>
people continuously mention it I wonder if I missed something about<br>
py.test's functionality that could be really useful and offset the<br>
speed problems.<br>
<br>
So, what are your reasons for using py.test (or some other test runner) ? :)<br>
<br></blockquote><div>We (at work) were using something different and find issues that py.test was</div><div>able to dismiss right away:</div><div><br></div><div>* no more failed tests for no apparent reasons (e.g. not exceptions but test runner errors)</div>
<div>* memory usage went considerably down, don't have exact numbers but I think at least by 20% less</div><div>* scenarios were total isolation was needed; py.test provided a "boxed" option to run a test in a separate thread.</div>
<div>* SPEED.</div><div><br></div><div>Ok, I know you say py.test is slower according to your numbers.In our specific case we have</div><div>over 800 tests, of those 800 at least 600 are database bound and *need* a schema loaded.</div>
<div><br></div><div>With the previous test runner we were seeing complete runs close to 15 minutes.</div><div><br></div><div>With py.test and the `xdist` plugin we were able to bring down that to under 5 minutes. The dev team was </div>
<div>jumping up and down once everything was running smoothly and under 5 minutes.</div><div><br></div><div>With that speed jump, we would not really care about tenths of a second to run a single test, but how to</div><div>
make everything as a whole to behave better.</div><div><br></div><div>If I would have to narrow it down, our team would most likely say these are killer for us:</div><div><br></div><div>* boxing tests</div><div>* distribution of tests</div>
<div><br></div><div>In my case, I love the huge outputs that I can fine-tune with verbosity control, and to be able to place</div><div>ipdb.set_trace() arbitrarily and not mess the test runner with stdout capturing (this is different than say `testrunner --pdb`)</div>
<div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
-Herman<br>
<br>
P.S. I know people tend to get sensitive about their tools, so I don't<br>
want to start a flame war, just genuinely curious why people are using<br>
this particular one..<br>
<br>
_______________________________________________<br>
testing-in-python mailing list<br>
<a href="mailto:testing-in-python@lists.idyll.org">testing-in-python@lists.idyll.org</a><br>
<a href="http://lists.idyll.org/listinfo/testing-in-python" target="_blank">http://lists.idyll.org/listinfo/testing-in-python</a><br>
</blockquote></div><br>