[TIP] [nose-dev] Re: [nose-users] nose2: the jazz odyssey begins

[jason pellerin]

On Fri, Jan 20, 2012 at 10:30 AM, Michael Foord
<michael at voidspace.org.uk> wrote:
> On 20/01/2012 15:18, Michael Foord wrote:
>>
>> On 20/01/2012 14:07, jason pellerin wrote:
>>>
>>> 2012/1/19 LiuCougar<liucougar at gmail.com>:
>>>>
>>>> Hi jason,
>>>> I don't see multiprocessing plugin being mentioned in the nose2 doc
>>>> anywhere. is it one of the things which won't be available in nose2? (I
>>>> could not find a list of plugins which can't be ported to nose2)
>>>
>>> Hi Heng,
>>>
>>> Multiprocessing is not there in nose2 yet but there definitely will be
>>> some kind of test distribution/parallelization plugin. Right now I'm
>>> still thinking about different ways to approach the problem and not
>>> ready to start coding yet. If anything it should be easier with
>>> nose2's plugin system and non-lazy test collection, but "easier" isn't
>>> "easy" ...
>>
>>
>> I did some thinking about test parallelization for unittest2.
>>
>> The test runner should be able to be run as a master or slave with a
>> protocol for communicating results from slave to master. The master does
>> test collection and launches slaves (either locally or remotely) with the
>> same configuration as the master (plugin state, command line args etc as
>> appropriate).
>>
>> Slaves then ask the master for tests and communicates the results back.
>> The master creates the test report. All objects attached to test reports (by
>> plugins on the slave) must be serializable.
>>
>
> This would all be done in as a plugin (naturally). I specifically included a
> hook point that allowed a plugin to completely take over the running of
> tests for this purpose.
>
> I had in mind using subprocesses rather than multiprocessing specifically.
> So I'm not entirely sure we're even talking about the same topic... Oh well.
> :-)

We are! Your approach above is basically what I've been thinking of as
"option A" (even down to using subprocess or execnet instead of
multiprocessing to launch the slaves). Option B, which I'm still
thinking about so it may be impossible or impractical, would be to use
concurrent.futures and a much dumber slave side that just runs test
cases/suites and records the hooks and events they want to fire, then
sends those back to the master as the future's return value. The
master then "plays the tape" to fire the real hooks with real events.
Might work, might be nice and simple... haven't thought it through
yet.

JP