[TIP] Problem collecting coverage for D-Bus activated subprocesses

Ned Batchelder ned at nedbatchelder.com
Sat Aug 9 08:01:39 PDT 2014


Sorry it's taken me a few days to read this closely, there's lots of 
good information here!  More below...

On 8/6/14 7:54 PM, Barry Warsaw wrote:
> On Aug 05, 2014, at 05:51 PM, Ned Batchelder wrote:
>
>> Could it be due to an os._exit() after forking?
> It wasn't, but this was a great clue.  It made me think about the atexit
> handling for writing out the data.  I instrumented the atexit callback to get
> some debugging output and that led me to two problems.
>
> The first indeed was that the D-Bus subproc's working directory was / which of
> course wasn't writable.  Instead of os.chdir()'ing I set COVERAGE_FILE in my
> tox.ini file and that got properly inherited by the subprocs.
>
> After fixing that and instrumenting the atexit function, I realized that the
> subprocs actually *were* writing their .coverage.* files, but they were
> getting erased when the main process (i.e. the one running the nose2 runner)
> exited.  This seemed odd to me, but then I remember that I'd been using the
> nose2_cov plugin so I peeked at its code.  It calls coverage.erase() before it
> calls coverage.start() and that does seem to be the source of the problem.  If
> I stop using nose2_cov and instead just enable coverage in my application's
> own nose2 plugin - which doesn't call .erase() - then everything works.
> Mostly[*].
The nose plugins are frankly a bane of my existence, because they are 
very opinionated, and I don't use them so I often overlook the 
possibility of their effects.  I prefer to control coverage directly.

>
> That's probably not the whole story, given that an .erase() call before a
> .start() call in the main process shouldn't erase all the data files at the
> *end* of the test suite run.  That's what appears to be happening, though now
> that things are working I may not investigate further.

I wouldn't expect erase to erase at the end, and I wouldn't expect it to 
erase all the parallel files.  Hmm, mysterious.

>
> [*] I have noticed a few anomalies:
>
> My D-Bus subprocs are invoked by the .conf file via `$python -m
> package.module` and while that module's main() starts coverage quite early on,
> the file itself doesn't show up at all in the coverage reports, and some of
> the imports, decorators, and other such bits are not covered at all.  Modules
> *it* calls does though.  I don't think my [run]omit section is filtering them
> out.

It is a quirk of the CPython trace function that it only takes effect 
when a function is called.  So you call coverage.start(), then it isn't 
until the next function call in your code that the tracing will begin.

>
> I run all this under tox and the D-Bus subprocs are executed out of the local
> .tox/<env>/lib/python3.4/site-packages directory, so I have to have a [paths]
> section to make equivalent those file paths and the in-tree paths, e.g.:
>
> [paths]
> source =
>      systemimage
>      .tox/coverage/lib/python*/site-packages/systemimage
>
> I wonder if that's common enough for folks running tests via tox.
>
> I get a little coverage on pkg_resources inside the tox directory, and this
> cannot be [run]omit-d out of the results, even though I have:
>
> omit =
>       setup*
>       systemimage/data/*
>       systemimage/docs/*
>       systemimage/testing/*
>       systemimage/tests/*
>       /usr/lib/*
>       .tox/coverage/lib/python*/site-packages/pkg_resources
>
> No idea what's happening there.  It does skew the numbers though, given that
> that file is only covered 2% while almost all other files are in the 70-90%
> range (only two are otherwise below 50%).  It brings total coverage down to
> 41%.
>
> So, progress is good, but I want to spend some time further investigating to
> be sure I'm getting accurate coverage for everything else.  Thanks very much
> for the help!
>
> Cheers,
> -Barry
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20140809/d87342db/attachment.htm>


More information about the testing-in-python mailing list