[TIP] why do you use py.test?

[Alfredo Deza]

On Mon, Feb 14, 2011 at 8:12 PM, Herman Sheremetyev <herman at swebpage.com>wrote:

> Hey everyone,
>
> I've seen a lot of messages on this list of people mentioning they use
> py.test, so I'm curious what advantages it offers. I gave it a try and
> immediately noticed it was literally 10x slower than nose, taking 0.60
> seconds to run the same set of tests it takes nose 0.062. As a result
> I pretty much wrote it off since test speed is pretty important to me,
> and all the red in my terminal seemed a bit unnecessary. But as I see
> people continuously mention it I wonder if I missed something about
> py.test's functionality that could be really useful and offset the
> speed problems.
>
> So, what are your reasons for using py.test (or some other test runner) ?
> :)
>
> We (at work) were using something different and find issues that py.test
was
able to dismiss right away:

* no more failed tests for no apparent reasons (e.g. not exceptions but test
runner errors)
* memory usage went considerably down, don't have exact numbers but I think
at least by 20% less
* scenarios were total isolation was needed; py.test provided a "boxed"
option to run a test in a separate thread.
* SPEED.

Ok, I know you say py.test is slower according to your numbers.In our
specific case we have
over 800 tests, of those 800 at least 600 are database bound and *need* a
schema loaded.

With the previous test runner we were seeing complete runs close to 15
minutes.

With py.test and the `xdist` plugin we were able to bring down that to under
5 minutes. The dev team was
jumping up and down once everything was running smoothly and under 5
minutes.

With that speed jump, we would not really care about tenths of a second to
run a single test, but how to
make everything as a whole to behave better.

If I would have to narrow it down, our team would most likely say these are
killer for us:

* boxing tests
* distribution of tests

In my case, I love the huge outputs that I can fine-tune with verbosity
control, and to be able to place
ipdb.set_trace() arbitrarily and not mess the test runner with stdout
capturing (this is different than say `testrunner --pdb`)





> -Herman
>
> P.S. I know people tend to get sensitive about their tools, so I don't
> want to start a flame war, just genuinely curious why people are using
> this particular one..
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20110214/dcde7b28/attachment.htm>