[TIP] [nose-dev] Re: [nose-users] nose2: the jazz odyssey begins

[Michael Foord]

On 20/01/2012 15:18, Michael Foord wrote:
> On 20/01/2012 14:07, jason pellerin wrote:
>> 2012/1/19 LiuCougar<liucougar at gmail.com>:
>>> Hi jason,
>>> I don't see multiprocessing plugin being mentioned in the nose2 doc
>>> anywhere. is it one of the things which won't be available in nose2? (I
>>> could not find a list of plugins which can't be ported to nose2)
>> Hi Heng,
>>
>> Multiprocessing is not there in nose2 yet but there definitely will be
>> some kind of test distribution/parallelization plugin. Right now I'm
>> still thinking about different ways to approach the problem and not
>> ready to start coding yet. If anything it should be easier with
>> nose2's plugin system and non-lazy test collection, but "easier" isn't
>> "easy" ...
>
> I did some thinking about test parallelization for unittest2.
>
> The test runner should be able to be run as a master or slave with a 
> protocol for communicating results from slave to master. The master 
> does test collection and launches slaves (either locally or remotely) 
> with the same configuration as the master (plugin state, command line 
> args etc as appropriate).
>
> Slaves then ask the master for tests and communicates the results 
> back. The master creates the test report. All objects attached to test 
> reports (by plugins on the slave) must be serializable.
>

This would all be done in as a plugin (naturally). I specifically 
included a hook point that allowed a plugin to completely take over the 
running of tests for this purpose.

I had in mind using subprocesses rather than multiprocessing 
specifically. So I'm not entirely sure we're even talking about the same 
topic... Oh well. :-)

Michael

> There are issues around fixtures for parallelized tests, but I'd be 
> inclined to punt and say that ensuring a test suite *can* be 
> parallelized is up to the test writer. Remote slaves creates a whole 
> new bunch of issues (do you package up and send all the test code to 
> the slave, how do you ensure dependencies are in place, what if some 
> of the slaves are on different platforms and can't run all the tests 
> etc). py.execnet has solved many of those problems, so either using it 
> or borrowing liberally would be wise.
>
> All the best,
>
> Michael Foord
>
>>
>> The things that won't work in nose2 are basically the ones in nose
>> that depend on test collection being lazy: weird project layouts where
>> modules shadow each other, and plugins like the isolation plugin that
>> assume a test module is imported and run before other test modules are
>> imported.
>>
>> JP
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>
>


-- 
http://www.voidspace.org.uk/

May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html