[TIP] [py-dev] yield-tests and fixtures: should they have a future?

[holger krekel]

Hi Ronny,

On Fri, Dec 21, 2012 at 11:37 +0100, Ronny Pfannschmidt wrote:
> Hi all,
> 
> My oppinion on yield Tests is,
> that they should be turned into part of reporting extensions
> instead of their current place at running/collection
> 
> i would like to have more than one report for functional/acceptance
> tests anyway preferably in a way that allows parts to fail but still
> run the complete test
>
> with that in place a yield test would be just a loop running the check 
> of all items without propagating single item failures to the outside

Well, one of the main reasons people used "yield" is to get separate progress
dots, and to allow some callables to fail without disrupting the whole sequence.
This means that the running of single callables needs to be reported separately.
Let's put detailed discussions of this to #pylib rather, it's going to get
pytest-specific.  I am not sure yet, also taking Jason's confirmation
into account, supporting yield is worth much.  Especially since pytest already
has extensive means for parametrization.

> also there is another upcoming task
> yield tests based on the the new async api

that's not materialized yet i think.

holger

> 
> best,
> Ronny
> 
> On 12/21/2012 10:51 AM, holger krekel wrote:
> > Hi testing folks, hi Jason,
> >
> > i am looking at some recent pytest issues and would like to simplify
> > pytest's internal fixture handling.  One obstacle/complication are
> > yield-tests, i.e. the style of producing tests with a generator::
> >
> >      def test_gen(self):
> >          for x in range(10):
> >              yield check, x
> >
> > This currently produces 10 test items, calling the "check" function with the
> > respective parameter.  This by itself is not a big deal to support.
> > However, some people expect fixtures/setup_function/method functions to execute
> > before the generator does and this mixes the collection with the runtest phase.
> > Unfortunately, nose also supports this notion although i am wondering how
> > nose2 is going to deal with it as Jason also plans to separate collection
> > from running.
> >
> > So i am thinking about dropping fixture/setup support for yield-tests in pytest
> > but OTOH i'd like to keep backward and nose compatibility.  As far as pytest
> > is concerned, it has many others means of parametrization independently
> > from yield, see e. g. http://pytest.org/latest/fixture.html#fixture-parametrize
> > and http://pytest.org/latest/parametrize.html and more and more people
> > are starting to use those (pytest does not document or recommend yield
> > for 1-2 years now).
> >
> > If anyone has any input/thoughts on this, please shoot.
> >
> > best,
> > holger
> >
> > _______________________________________________
> > testing-in-python mailing list
> > testing-in-python at lists.idyll.org
> > http://lists.idyll.org/listinfo/testing-in-python
> 
> _______________________________________________
> py-dev mailing list
> py-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/py-dev
>