[TIP] Branch report and subprocess coverage using Ned's coverage.py in Hudson matrix build job

[Ned Batchelder]

  Olemis,

I'll take this off-list with you...

--Ned.

On 9/22/2010 9:30 AM, Olemis Lang wrote:
> Hi !
>
> :o)
>
> On Tue, Sep 21, 2010 at 1:45 PM, Olemis Lang<olemis at gmail.com>  wrote:
>> On Tue, Sep 21, 2010 at 1:10 PM, Ned Batchelder<ned at nedbatchelder.com>  wrote:
>>> On 9/21/2010 11:57 AM, Olemis Lang wrote:
> [...]
>>>> OTOH I'm facing some problems setting up subprocess coverage .
>> [...]
>>>> export COVERAGE_PROCESS_START=.coveragerc
>>>>
>> [...]
>>>> Q:
>>>>    - What could I do ?
>>> You should verify that the subprocess run by trac is using the Python
>>> installation and Python path that you've modified.  You also need to verify
>>> that it sees the COVERAGE_PROCESS_START environment variable.
>> I'd expect this to happen since I included the export shown above .
>> Nonetheless I'll check out ...
>>
>>> Can you
>>> modify standalone.py just to get some debugging information from it?  Have
>>> it dump out the sys.executable, sys,path, and os.environ to see what it
>>> sees.
>>>
>> Of course !!! Let's see what happens .
>>
> [...]
>> Thnx !
>> I'll let you know about progress I make .
>>
> Here you have the (relevant) details of the execution context. I
> just removed some environment variables I considered irrelevant
> (e.g. build job parameters ... did I already say that the jobs
> have further parameters besides the axes ? ) so if you
> need further details or expect something to be in env vars ,
> please tell me ;o)
>
> I also obfuscated path info using Hudson vars so that it'd be
> easier to know the details. Further hints
>
>    - The path $PYTHON_DEPS is a folder
>      that contains dependencies needed to compile and execute the
>      build jobs. They are installed (most of the time) using
>      `setup.py develop` ;o) .
>    - $WORKSPACE/venv/$BUILD_NUMBER is created using `virtualenv`
>      (yes! using --no-site-packages ;o)
>
> {{{
>
> **********************************************************************
> * sys.executable
> **********************************************************************
> $WORKSPACE/venv/$BUILD_NUMBER/bin/python2.5
>
> **********************************************************************
> * sys.path
> **********************************************************************
> ['$PYTHON_DEPS/Trac-0.11.7/trac/web',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5/site-packages/pip-0.7.2-py2.5.egg',
>   '$PYTHON_DEPS/twill-0.9',
>   '$PYTHON_DEPS/unittest-xml-reporting-1.0.3/src',
>   '$PYTHON_DEPS/ciutils',
>   '$PYTHON_DEPS/coverage-3.4',
>   '$PYTHON_DEPS/Babel',
>   '$PYTHON_DEPS/Genshi',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5/site-packages/setuptools-0.7a1dev_r84272-py2.5.egg',
>   '$PYTHON_DEPS/Trac-0.11.7',
>   '$PYTHON_DEPS/simplejson-2.0.9',
>   '$WORKSPACE/trunk',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5/plat-linux2',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5/lib-tk',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5/lib-dynload',
>   '/usr/lib/python2.5',
>   '/usr/lib/python2.5/plat-linux2',
>   '/usr/lib/python2.5/lib-tk',
>   '$WORKSPACE/venv/$BUILD_NUMBER/lib/python2.5/site-packages']
>
> **********************************************************************
> * os.environ
> **********************************************************************
> LINES = 24
> WORKSPACE = $WORKSPACE
> PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
> SHELL = /bin/bash
> COVERAGE_PROCESS_START = .coveragerc
> BUILD_NUMBER = $BUILD_NUMBER
> BUILD_REPORT = $WORKSPACE/
> UPSTART_EVENTS = runlevel
> UPSTART_JOB = rc
> PLAT = linux-i686
> _ = $WORKSPACE/venv/$BUILD_NUMBER/bin/coverage
> VERBOSE = no
> OLDPWD = $WORKSPACE
> PWD = $WORKSPACE/trunk
> }}}
>
> Further comments :
>
>    - `Source code is unavailable. Some possible reasons are:`
>      Would be nice to add support for this in the near future
>      in order to see the source code with coverage annonations
>      (I can't wait to show this to my bozz :D ).
>      This is something I could help with ;o) just don't know
>      exactly what needs to be done and how.
>    - After adding parallel=true to .coveragerc a single file is
>      still generated, so no coverage tracking happens in
>      subprocess .
>    - I got the fabulous greeny&  pinky bar-like little
>      thingies showing branch coverage info (YES!!!).
>      Nevertheless they're so pink ... I need the
>      subprocesses thing working !!! :o)
>    - `--omit` seemed to be working with 3.3.1 but it
>      doesn't if using 3.4 .
>    - A question: does it make sense to include tests top
>      level package in coverage report ? I mean it's
>      possible that some paths in testing code won't be
>      executed (e.g. platform-specific test setup) and
>      that should be ok.
>    - Finally, if a module is named `api.py` then there's
>      a page like `[...]/cobertura/tracrpc/api_py/`. In the
>      table named `Coverage Breakdown by Class` there is a link
>      to `[...]/cobertura/tracrpc/api_py/api/` that should
>      show `Class Coverage summary` for this module. Nevertheless
>      it leads to `Remote API` page. The fact is that /api handle
>      after every valid Hudson page is used to retrieve the
>      data in different formats (kind of RESTful interface).
>      Would it be a good idea to tweak something inside
>      `coverage.py` so as to work around this situation ?
>      I'll ask to hudson-users as well to know
>      if there is another way to see its contents. Probably
>      they'll assume Hudson's responsability ;o).
>
> I'll be looking forward to your comments .
> Thnx in advance !
>