[TIP] Branch report and subprocess coverage using Ned's coverage.py in Hudson matrix build job

[Olemis Lang]

Hi !

I come back once again with the subject of coverage + CI with Hudson.

I run Ned's coverage with `--branch` option set . After generating
Cobertura XML report it includes tags similar the one shown below :

{{{
#!xml

[...]
  <package branch-rate="0.0230263157895" complexity="0.0"
line-rate="0.29704797048" name="tracrpc">
    <classes>
      <class branch-rate="0.0" complexity="0.0"
filename="tracrpc/__init__.py" line-rate="0.857142857143"
name="__init__">
        <methods/>
        <lines>
          <line hits="1" number="2"/>
          [...]
          <line hits="0" number="24"/>
        </lines>
      </class>
      <class branch-rate="0.0806451612903" complexity="0.0"
filename="tracrpc/api.py" line-rate="0.337579617834" name="api">
        <methods/>
        <lines>
          <line hits="1" number="2"/>
          [...]
          <line branch="true" hits="0" number="298"/>
          <line branch="true" hits="0" number="299"/>
          [...]
          <line hits="0" number="346"/>
        </lines>
      </class>
      [...]
    </classes>
  </package>
[...]
}}}

So I suppose it's tracking branch coverage. Nonetheless the HTML report
published by Hudson displays (under Conditionals) `N/A` and `0/0`
rather than meaningful results.

Q:
  - Is it the expected behavior or probably I'm missing something?
  - AYCS `method` tag is always empty. Is it Ok ? Is there a way
    to make coverage output method info ?

OTOH I'm facing some problems setting up subprocess coverage . I
read the instructions mentioned in the tutorial [1]_ and tried to
setup something like this (assume there's a virtualenv under
$WORKSPACE/venv/$BUILD_NUMBER/ folder, and coverage is already
installed in there) :

{{{
#!/bin/bash

echo "**********************************************************"
echo "* Preparing for subprocess coverage "
echo "**********************************************************"

echo "import coverage; coverage.process_startup()" >
$WORKSPACE/venv/$BUILD_NUMBER/lib/python$PYTHON/site-packages/sitecustomize.py
echo "[run]"       >  .coveragerc
echo "branch=true" >> .coveragerc
export COVERAGE_PROCESS_START=.coveragerc

echo "**********************************************************"
echo "* Running tests with coverage "
echo "**********************************************************"

$WORKSPACE/venv/$BUILD_NUMBER/bin/coverage run --branch -p setup.py
test -r ciutils:junitrunner

echo "**********************************************************"
echo "* Files produced by coverage "
echo "**********************************************************"

ls -la | grep coverage

echo "**********************************************************"
echo "* Combining coverage data "
echo "**********************************************************"

$WORKSPACE/venv/$BUILD_NUMBER/bin/coverage combine

echo "**********************************************************"
echo "* Generating Cobertura XML report "
echo "**********************************************************"

$WORKSPACE/venv/$BUILD_NUMBER/bin/coverage xml
--omit=$WORKSPACE/venv/$BUILD_NUMBER,[...]

}}}

The output contains this snippet so I suppose -p is working

{{{
**********************************************************
* Files produced by coverage
**********************************************************
-rw-r--r-- 1 hudson nogroup   52 2010-09-20 23:55 .coverage.mypc.7168.064897
-rw-r--r-- 1 hudson nogroup   18 2010-09-20 23:53 .coveragerc
}}}

Nevertheless , considering the fact that there's a single file and two
processes
it seems that subprocess doesn't track script execution .

BTW , the subprocess is launched this way (so it might
not be a subprocess strictly speaking, or is it ? )

{{{
#!py

server = Popen([exe, os.path.join(self.trac_src,
       'trac', 'web',
       'standalone.py')] + options + [self.tracdir],
       stdout=logfile, stderr=logfile,
       close_fds=close_fds,
       cwd=self.command_cwd,
      )
}}}

In that case exe should be set to
`sys.executable` so I suppose virtualenv
sets this value to
$WORKSPACE/venv/$BUILD_NUMBER/bin/python$PYTHON
isn't it ?
The fact is that I can't do anything about that
since that code is inside Trac source code (in
`trac.tests.functional.testenv` to be more
precise ;o).

Q:
  - What could I do ?

Any help will be appreciated . Thnx in advance !

.. [1] Measuring subprocesses
        (http://nedbatchelder.com/code/coverage/subprocess.html)

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article: