[TIP] Problem collecting coverage for D-Bus activated subprocesses

[Barry Warsaw]

On Aug 05, 2014, at 05:51 PM, Ned Batchelder wrote:

>Could it be due to an os._exit() after forking?

It wasn't, but this was a great clue.  It made me think about the atexit
handling for writing out the data.  I instrumented the atexit callback to get
some debugging output and that led me to two problems.

The first indeed was that the D-Bus subproc's working directory was / which of
course wasn't writable.  Instead of os.chdir()'ing I set COVERAGE_FILE in my
tox.ini file and that got properly inherited by the subprocs.

After fixing that and instrumenting the atexit function, I realized that the
subprocs actually *were* writing their .coverage.* files, but they were
getting erased when the main process (i.e. the one running the nose2 runner)
exited.  This seemed odd to me, but then I remember that I'd been using the
nose2_cov plugin so I peeked at its code.  It calls coverage.erase() before it
calls coverage.start() and that does seem to be the source of the problem.  If
I stop using nose2_cov and instead just enable coverage in my application's
own nose2 plugin - which doesn't call .erase() - then everything works.
Mostly[*].

That's probably not the whole story, given that an .erase() call before a
.start() call in the main process shouldn't erase all the data files at the
*end* of the test suite run.  That's what appears to be happening, though now
that things are working I may not investigate further.

[*] I have noticed a few anomalies:

My D-Bus subprocs are invoked by the .conf file via `$python -m
package.module` and while that module's main() starts coverage quite early on,
the file itself doesn't show up at all in the coverage reports, and some of
the imports, decorators, and other such bits are not covered at all.  Modules
*it* calls does though.  I don't think my [run]omit section is filtering them
out.

I run all this under tox and the D-Bus subprocs are executed out of the local
.tox/<env>/lib/python3.4/site-packages directory, so I have to have a [paths]
section to make equivalent those file paths and the in-tree paths, e.g.:

[paths]
source =
    systemimage
    .tox/coverage/lib/python*/site-packages/systemimage

I wonder if that's common enough for folks running tests via tox.

I get a little coverage on pkg_resources inside the tox directory, and this
cannot be [run]omit-d out of the results, even though I have:

omit =
     setup*
     systemimage/data/*
     systemimage/docs/*
     systemimage/testing/*
     systemimage/tests/*
     /usr/lib/*
     .tox/coverage/lib/python*/site-packages/pkg_resources

No idea what's happening there.  It does skew the numbers though, given that
that file is only covered 2% while almost all other files are in the 70-90%
range (only two are otherwise below 50%).  It brings total coverage down to
41%.

So, progress is good, but I want to spend some time further investigating to
be sure I'm getting accurate coverage for everything else.  Thanks very much
for the help!

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20140806/409e4185/attachment.pgp>