[bip] Announcing PEBL (and asking for assistance)

[Kevin Teague]

On 17-Jun-08, at 2:34 PM, Abhik Shah wrote:

> Installation is currently a bit painful but I'm working on  
> improving that.

Yes, this is common with a great number of scientific toolkits and  
apps ...

... I don't follow the science to closely, so I'll only make comments  
regarding installation:

---
You can't have imports in setup.py that are required to install by  
setup.py. e.g.:

    import numpy
    setup(
     install_requires=[
         'numpy >= 1.0.3',       # matrices, linear algebra, etc

setup.py won't run unless you've already installed numpy here, but  
you need to run "include_dirs=[numpy.get_include()]" to compile the  
software.

---
Installing software using easy_install works OK for utility packages  
(e.g. nose), but it can get nasty real quick when using it to install  
application software, since code get's installed on Python's path -  
e.g. if you install Application A and it uses Library B v1.0, and  
Application C is installed and it uses Library B v2.0, next time you  
use Application A you will find it borked :(

If you add instructions for VirtualEnv you can isolate your  
applications installation from the plethora of other applications a  
user may wish to run against their Python interpreter:

http://pypi.python.org/pypi/virtualenv

With VirtualEnv, you can run easy_install without risking breaking  
other previously installed applications.

However, if you have custom build and install steps, you will  
probably want to go beyond hacking on setup.py and use a proper build  
tool. There are quite a few build tools written in Python (http:// 
wiki.python.org/moin/ConfigurationAndBuildTools). My current favorite  
is Buildout (http://pypi.python.org/pypi/zc.buildout) with this you  
would:

  1. Add a buildout.cfg file to the root of your project directory.

  2. Write Buildout Recipes for any custom Python code to automate  
manual installation parts (e.g. make the numpy include files  
available for compilation somewhere).

End users could then:

  1. Download your project

  2. Bootstrap the build tool: "python2.5 bootstrap.py"

  3. Run the build, which will download, compile and install all the  
project dependencies: "./bin/buildout"

  4. You may have an alternate build config used for development,  
e.g. "./bin/buildout -c dev.cfg" (would install nose for example)

Everything built by buildout gets installed inside your project  
directory. This is great because if you are just trying something  
out, you don't have to rely on an uninstallation script, or worry  
about leaving a bunch of packages in your site-packages directory.

However, you may find the current documentation for Buildout somewhat  
difficult to get started with. Also, you'll probably find a number of  
packages you depend upon which don't install so easily from source as  
a Python egg so easily (e.g. matplotlib). For trickier packages, I  
sometimes just compile a binary egg, throw it into an Apache  
directory, and add the URL to the project's buildout.cfg find-links  
directive - this gives the build an additional location beyond PyPI  
where it can fetch packages from - e.g. http://www.bcgsc.ca/downloads/ 
parts/software/resources/eggs/

I am using Buildout for Zope, Plone and Grok based projects, whose  
server-side applications can have fairly complex deployments (lots  
and lots of eggs and getting the specific versions of the eggs just  
right for each deployment is very important). When you can walk up to  
a project you've never used with a complex deployment and run "./bin/ 
buildout" and get a complete working development environment or  
application environment, it's a beautiful thing :)