[TIP] distributed testing fixes (execnet-1.0.8 and a green experiment)
holger at merlinux.eu
Fri Jul 30 15:41:25 PDT 2010
see below for one and a half execnet releases which fix some
issues that were originally registered as py.test ones,
namely rsync issues with symbolic links and permissions.
You can do "pip -U execnet" or "pip -U pytest-xdist" to make sure
you have the latest of everything. ("easy_install" is fine as well).
----- Forwarded message from holger krekel <holger at merlinux.eu> -----
Date: Sat, 31 Jul 2010 00:34:04 +0200
From: holger krekel <holger at merlinux.eu>
To: execnet-dev at codespeak.net,
python announce <python-announce-list at python.org>
User-Agent: Mutt/1.5.17+20080114 (2008-01-14)
Subject: [execnet-dev] execnet-1.0.8 and a green experiment
I just released execnet-1.0.8, see the Changelog for details, below.
What is execnet?
execnet is a small pure python library that manages distributed code execution
across Python versions and host boundaries. It provides a self-bootstrapping
share-nothing model with channels to send/receive data messages. It also
helps with managing a cluster of (local or remote) processes and comes
with tons of examples and API docs:
green version of execnet
For those interested i hacked up a green version of execnet
which is API-compatible (and less stable) than the "main"
version which relies on operating system threads. The green version
requires the eventlet library and is thus not fully
self-bootstrapping (you need to install eventlet on all remote nodes
you want to execute code on). There is no release but you can peruse
and "python setup.py develop" to play with it. It will never spawn
any operating system threads. Rather remote_exec() will spawn
greenlets/micro-threads of which you can easily have a few thousand.
It probably only is stable enough for simple scenarios at this point.
Changelog 1.0.8 (compared to 1.0.7):
- new ``gateway.remote_exec(func, **kwargs)`` style fo executing
a pure function with parameters. The function on the remote
side also needs to accept a ``channel`` which allows it to
communicate back and forth. Thanks to Ronny Pfannschmidt
for implementing it with special kudos to Maciej Fijalkowski
for writing a "pure-function" checker so that on Python2.6
onwards non-pure functions will be rejected.
- enhance rsyncing to also sync permissions (stat().st_mode)
of directories and files.
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/68/)
- fix rsyncing of symlinks, thanks to Charles Solar
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/70/)
- update internal usage of apipkg to 1.0b6
- remote_exec(module) now makes sure that the linecache is updated
before reading and sending the source. thanks Ronny, Matt.
- removed all trailing whitespace from source files
execnet-dev mailing list
execnet-dev at codespeak.net
----- End forwarded message -----
More information about the testing-in-python