[TIP] StaticPython, anyone?
peter at psftw.com
Wed Jul 13 18:02:11 PDT 2011
I find StaticPython to be very interesting:
Google Code project page:
(check out the slides at the bottom if interested)
I work in software QA and use Python and various libraries to
test/deploy/automate Java applications. Installing a basic set of
Python packages across various Linux machines in order to have a
consistent testing rig has become a chore, especially when libraries
require C extensions.
First, I took the approach of figuring out and then documenting the
steps to install specific libraries against the OS-bundled Python
interpreter, but then this approach quickly becomes a mess due to
* Python version compatibilities (before I heard of tox)
* The cheeseshop makes it difficult to reproduce exact-version installs
since fast moving libraries don't provide old versions at some point.
Even with a hacks like pip requirements or build out, it is still a
chore in my opinion.
* Various OS quirks: Does it have a the right version of a dependent
library? Does it have development headers or is that a separate
package? In the case where the OS doesn't include a library or headers,
can I compile it myself from source, and does that source have yet more
dependencies? Does it contain a working compile toolchain? Can
pip/easy_install use it?
* etc ...
My next approach was to manually build a *quasi-static Python and
whatever libraries I needed from source on a per architecture basis,
which mostly works, but adds a lot of complexity and when something
doesn't work quite right I pull out my hair. It also requires shipping
a ~75MB bundle.
*I could never figure out how to remove the libc dependency, so I also
had an specific-version-of-libc dependency!
My new approach will likely be stripping out all non-pure Python code
that I rely on and move to StaticPython. Since most of the libraries I
use are pure Python already, it shouldn't be too painful. In addition
to StaticPython, I'm exploring Jython since I am almost guaranteed to
have a working JVM on my target systems. [Most of the libraries I use
with C extensions are DB related, such as cx_Oracle and pymssql, which
could be replaced with JDBC drivers in theory]
My experience with StaticPython so far has gone well. The only issue
I've encountered is that some of my code that depends on threading
crashes with a segmentation fault when running 50+ threads -- I haven't
Thoughts about my approach and/or StaticPython in general?
More information about the testing-in-python