[TIP] distributed testing fixes (execnet-1.0.8 and a green experiment)
holger krekel
holger at merlinux.eu
Fri Jul 30 15:41:25 PDT 2010
hi all,
see below for one and a half execnet releases which fix some
issues that were originally registered as py.test ones,
namely rsync issues with symbolic links and permissions.
You can do "pip -U execnet" or "pip -U pytest-xdist" to make sure
you have the latest of everything. ("easy_install" is fine as well).
best,
holger
----- Forwarded message from holger krekel <holger at merlinux.eu> -----
Date: Sat, 31 Jul 2010 00:34:04 +0200
From: holger krekel <holger at merlinux.eu>
To: execnet-dev at codespeak.net,
python announce <python-announce-list at python.org>
User-Agent: Mutt/1.5.17+20080114 (2008-01-14)
Subject: [execnet-dev] execnet-1.0.8 and a green experiment
I just released execnet-1.0.8, see the Changelog for details, below.
What is execnet?
---------------------
execnet is a small pure python library that manages distributed code execution
across Python versions and host boundaries. It provides a self-bootstrapping
share-nothing model with channels to send/receive data messages. It also
helps with managing a cluster of (local or remote) processes and comes
with tons of examples and API docs:
http://codespeak.net/execnet
green version of execnet
--------------------------------
For those interested i hacked up a green version of execnet
which is API-compatible (and less stable) than the "main"
version which relies on operating system threads. The green version
requires the eventlet library and is thus not fully
self-bootstrapping (you need to install eventlet on all remote nodes
you want to execute code on). There is no release but you can peruse
http://code.google.com/p/execnet-green/source/checkout
and "python setup.py develop" to play with it. It will never spawn
any operating system threads. Rather remote_exec() will spawn
greenlets/micro-threads of which you can easily have a few thousand.
It probably only is stable enough for simple scenarios at this point.
best,
holger
Changelog 1.0.8 (compared to 1.0.7):
- new ``gateway.remote_exec(func, **kwargs)`` style fo executing
a pure function with parameters. The function on the remote
side also needs to accept a ``channel`` which allows it to
communicate back and forth. Thanks to Ronny Pfannschmidt
for implementing it with special kudos to Maciej Fijalkowski
for writing a "pure-function" checker so that on Python2.6
onwards non-pure functions will be rejected.
- enhance rsyncing to also sync permissions (stat().st_mode)
of directories and files.
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/68/)
- fix rsyncing of symlinks, thanks to Charles Solar
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/70/)
- update internal usage of apipkg to 1.0b6
- remote_exec(module) now makes sure that the linecache is updated
before reading and sending the source. thanks Ronny, Matt.
- removed all trailing whitespace from source files
_______________________________________________
execnet-dev mailing list
execnet-dev at codespeak.net
http://codespeak.net/mailman/listinfo/execnet-dev
----- End forwarded message -----
--
More information about the testing-in-python
mailing list