[TIP] Testify

[holger krekel]

On Sat, Oct 30, 2010 at 16:52 -0500, Kumar McMillan wrote:
> On Fri, Oct 29, 2010 at 4:36 PM, Rhett Garber <rhettg at gmail.com> wrote:
> >  * Split test suites into buckets for easy parallelization
> 
> Just like Holger, I too am very interested to hear about your approach
> to splitting up test suites for running in parallel.  Nose has a model
> for this but it's not very flexible.  py.test has a more flexible
> model.  There are still a lot of challenges with running tests in
> parallel and I think there is room for new ideas and strategies --
> especially since most typical ways of writing tests in Python end up
> sharing a lot of setup/teardown context which starts to make the
> mechanics of parallel testing very difficult.  Parallel tests leads to
> faster tests which leads to more productivity :)

Right, intelligent grouping is one difficulty.  Another is to have an extension
architecture that allows plugins to work in distributed situations as well.
E.g. coverage, profiling, junitxml-reporting etc.  Thirdly, when we distribute
tests to non-homogenous environments (instead of just subprocesses on a local
machine) there also is a deployment issue - how to prepare (remote) environments
so that we can run our tests properly? 

I believe that the last bit could be helped by making "tox" provide an API so
that we can reuse/extend tox.ini files to create environments, especially since
there is an upcoming "zero-installation" way now to bootstrap tox on any
machine. 

> That said, it would be a shame if there was another API split caused
> by a new test runner.  For that reason I'd encourage you to read up on
> Michael Foord's unittest2 work around formalizing a plugin API for
> test suites.  There are threads in the archives here.  Nose and
> py.test currently have a compatible plugin API (more or less) and this
> allows some sharing of 3rd party plugins between the two tools.  Nose
> also plans to be compatible with unittest2, not just for plugins but
> so that Nose can evolve into a thin layer on top of unittest2.

Well, I'd like to look into making py.test able to run unittest2 based
tests.  This way one could do distributed testing and use other bits for
unittest2 tests.  Maybe the reverse integration is also possible - though
i doubt it a bit because unittest2 does not really try as much as py.test 
to fit well for non-python tests and has different goals in some areas 
but maybe i am just lacking fantasy :)

cheers,
holger

> 
> And, yes, welcome to the list!  It's nice to see Yelp embracing the
> open source community more.
> 
> Kumar
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
> 

--