[TIP] massive parallel testing in python

[Kumar McMillan]

On Dec 5, 2007 10:44 AM, Noah Gift <noah.gift at gmail.com> wrote:
> >
> >
> > The good news is it definitely shortens the execution wall time,
> > depending
> > on how many resources you let it have.
>
> Paul,
>
> ATS looks like very interesting stuff.  I will take a look at it this
> week.  Thanks to everyone else, for the great suggestions.  On a
> related note, I would love to be involved in a testing related Sprint
> at PyCon.  I think they, the organizers, are still looking for Sprint
> leaders, so if someone is interested in doing more work on parallel
> testing in Python, I will show up.

this is a great idea for a sprint.  I've never led a sprint but I am
interested in doing so.  For it to work we would need a good backlog
of task tickets relating to the problem.  Off the top of my head, some
tasks:

 - benchmark worker-supervisor communication process (that which sends
test results over the wire)
 - benchmark speed of tests runs in parallel then benchmark speed of
same tests run in series, for comparison.  Do this for local and
remote workers.
 - add plenty of unit tests for all the various components involved
(this task needs to be more specific)
 - add plenty of functional tests for running tests in parallel on the
same machine, on remote machines, etc
 - add tests for many combinations of setup/teardown that could be
supported (at the package level, at the module level, etc)
 - perhaps implement XML output in nose core to address
worker-supervisor communication (see
http://code.google.com/p/python-nose/issues/detail?id=140 )
 - create capistrano (http://www.capify.org/) recipes that install
nose, plugins, and test suite to run on remote "worker" servers (or
implement in python, but this would be harder)

... most of these tasks assume we will have a proof of concept
implementation to start with.

-Kumar

> I suppose, I like the idea of
> having the ability to scale an arbitrary number of virtual machines as
> test nodes, and having the ability to control them to netboot a
> "clean" test operating system, build a database, do test, and then
> rebuild themselves.  That would be one heck of a Sprint.  What is
> great, is that if you have a firewire, or USB 2 drive, you can
> simulate something like this on your laptop, quite easily, and then
> roll it out into production.
>
>
>
> Noah
>