[TIP] Problem collecting coverage for D-Bus activated subprocesses
Ned Batchelder
ned at nedbatchelder.com
Sat Aug 9 08:01:39 PDT 2014
Sorry it's taken me a few days to read this closely, there's lots of
good information here! More below...
On 8/6/14 7:54 PM, Barry Warsaw wrote:
> On Aug 05, 2014, at 05:51 PM, Ned Batchelder wrote:
>
>> Could it be due to an os._exit() after forking?
> It wasn't, but this was a great clue. It made me think about the atexit
> handling for writing out the data. I instrumented the atexit callback to get
> some debugging output and that led me to two problems.
>
> The first indeed was that the D-Bus subproc's working directory was / which of
> course wasn't writable. Instead of os.chdir()'ing I set COVERAGE_FILE in my
> tox.ini file and that got properly inherited by the subprocs.
>
> After fixing that and instrumenting the atexit function, I realized that the
> subprocs actually *were* writing their .coverage.* files, but they were
> getting erased when the main process (i.e. the one running the nose2 runner)
> exited. This seemed odd to me, but then I remember that I'd been using the
> nose2_cov plugin so I peeked at its code. It calls coverage.erase() before it
> calls coverage.start() and that does seem to be the source of the problem. If
> I stop using nose2_cov and instead just enable coverage in my application's
> own nose2 plugin - which doesn't call .erase() - then everything works.
> Mostly[*].
The nose plugins are frankly a bane of my existence, because they are
very opinionated, and I don't use them so I often overlook the
possibility of their effects. I prefer to control coverage directly.
>
> That's probably not the whole story, given that an .erase() call before a
> .start() call in the main process shouldn't erase all the data files at the
> *end* of the test suite run. That's what appears to be happening, though now
> that things are working I may not investigate further.
I wouldn't expect erase to erase at the end, and I wouldn't expect it to
erase all the parallel files. Hmm, mysterious.
>
> [*] I have noticed a few anomalies:
>
> My D-Bus subprocs are invoked by the .conf file via `$python -m
> package.module` and while that module's main() starts coverage quite early on,
> the file itself doesn't show up at all in the coverage reports, and some of
> the imports, decorators, and other such bits are not covered at all. Modules
> *it* calls does though. I don't think my [run]omit section is filtering them
> out.
It is a quirk of the CPython trace function that it only takes effect
when a function is called. So you call coverage.start(), then it isn't
until the next function call in your code that the tracing will begin.
>
> I run all this under tox and the D-Bus subprocs are executed out of the local
> .tox/<env>/lib/python3.4/site-packages directory, so I have to have a [paths]
> section to make equivalent those file paths and the in-tree paths, e.g.:
>
> [paths]
> source =
> systemimage
> .tox/coverage/lib/python*/site-packages/systemimage
>
> I wonder if that's common enough for folks running tests via tox.
>
> I get a little coverage on pkg_resources inside the tox directory, and this
> cannot be [run]omit-d out of the results, even though I have:
>
> omit =
> setup*
> systemimage/data/*
> systemimage/docs/*
> systemimage/testing/*
> systemimage/tests/*
> /usr/lib/*
> .tox/coverage/lib/python*/site-packages/pkg_resources
>
> No idea what's happening there. It does skew the numbers though, given that
> that file is only covered 2% while almost all other files are in the 70-90%
> range (only two are otherwise below 50%). It brings total coverage down to
> 41%.
>
> So, progress is good, but I want to spend some time further investigating to
> be sure I'm getting accurate coverage for everything else. Thanks very much
> for the help!
>
> Cheers,
> -Barry
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20140809/d87342db/attachment.htm>
More information about the testing-in-python
mailing list