[TIP] why do you use py.test?

[Herman Sheremetyev]

Thanks for the explanations, everyone! I think I have a better grasp
on this now :)

(some comments inline)

On Tue, Feb 15, 2011 at 7:12 PM, holger krekel <holger at merlinux.eu> wrote:
> On Tue, Feb 15, 2011 at 10:12 +0900, Herman Sheremetyev wrote:
>> I've seen a lot of messages on this list of people mentioning they use
>> py.test, so I'm curious what advantages it offers. I gave it a try and
>> immediately noticed it was literally 10x slower than nose, taking 0.60
>> seconds to run the same set of tests it takes nose 0.062. As a result
>> I pretty much wrote it off since test speed is pretty important to me,
>> and all the red in my terminal seemed a bit unnecessary. But as I see
>> people continuously mention it I wonder if I missed something about
>> py.test's functionality that could be really useful and offset the
>> speed problems.
>
> first a note on speed.  py.test has higher overhead for executing a single
> tests than nose or unittest.  It spends more time in plugin hook invocations
> and could be optimized to be less noticeable.  But this hasn't been a priority
> because no one has yet told me it's of practical concern to them.
> Is it in your case?

Definitely. It seems like people that are using py.test are writing
much slower tests, perhaps exercising large portions of their
application in each one, so the extra setup time gets lost in the
noise. Whereas if you have a set of tests each of which takes
microseconds to run, running the test suite with unittest or nose
doesn't break you out of the development groove, but py.test takes so
long that I think it makes TDD style development much less practical.
At least that's the case for me. If I had to worry about doing
end-to-end testing in a large application, rather than doing quick
iterations in a small library, py.test sounds like it might make for a
good solution as it supports running the tests in parallel, which is
key when you have lots of large tests.

>> So, what are your reasons for using py.test (or some other test runner) ? :)
>
> i like many aspects of how it behaves and reports probably
> because i wrote much of it :)  I'd also be curious what people
> have to say why they prefer nose or other test runners, btw :)

These are just visual nitpicks, but I find the default informational
messages in the terminal being in red to be pretty confusing. Green,
or perhaps just neutral bold, would have been a better choice for
these I think. Also, the huge horizontal lines separating frames in
the stack traces are very hard to parse visually. This may just be me
though..

Cheers,

-Herman