[TIP] Test added (and executed) isn't increasing apparent coverage

Skip Montanaro skip.montanaro at gmail.com
Mon Apr 6 06:33:53 PDT 2020


> You need to also define the environment variable COVERAGE_PROCESS_START to point to the .coveragerc file to use.  You haven't mentioned it, so perhaps this is what is missing?

I wasn't aware a .coveragerc file was even required. I found no
tutorial so created a simple one:

[run]
branch = False
source = /home/skip/src/python/cpython/Lib/rattlesnake

[report]

then ran my make command. Still, only the first coverage command
produced debug output indicating my Lib/rattlesnake package would be
covered:

% COVERAGE_PROCESS_START=/home/skip/src/python/cpython/.coveragerc
/home/skip/.local/bin/coverage run --debug=trace -p
/home/skip/tmp/junk.py 2>&1 | grep rattlesnake
Tracing '/home/skip/src/python/cpython/Lib/rattlesnake/__init__.py'
Tracing '/home/skip/src/python/cpython/Lib/rattlesnake/converter.py'
Tracing '/home/skip/src/python/cpython/Lib/rattlesnake/opcodes.py'
Tracing '/home/skip/src/python/cpython/Lib/rattlesnake/blocks.py'
Tracing '/home/skip/src/python/cpython/Lib/rattlesnake/instructions.py'
Tracing '/home/skip/src/python/cpython/Lib/rattlesnake/util.py'

When I ran coverage over my unit tests it once again failed to even
mention the Lib/rattlesnake files:

% COVERAGE_PROCESS_START=/home/skip/src/python/cpython/.coveragerc
/home/skip/.local/bin/coverage run --debug=trace -p
./Tools/scripts/run_tests.py test_rattlesnake 2>&1 | grep rattlesnake
0:00:03 load avg: 3.60 [1/1] test_rattlesnake passed

I finally figured out that perhaps I couldn't use the command line at
all. Getting rid of the -p command line flag and setting "parallel =
True" in the config file caused things to start working. I don't know
if I'm a dolt or if this 100% reliance on config files when the
program being covered spawns subprocesses is something subtle, but I
certainly missed the note on this page about it:

https://coverage.readthedocs.io/en/coverage-5.0.4/subprocess.html

I tend to read the text. My brain views everything else as a sidebar
(even stuff in sky blue boxes which say "note"). Maybe the note should
be the very first thing in that section? Maybe there's a "warning"
directive which would make the note stand out more? Not complaining
that what's there is wrong, just that my brain filtered it out.

A bit more digging... During this process I stumbled on a section in
the Python dev guide about code coverage:

https://devguide.python.org/coverage/#measuring-coverage

There's no mention of parallel this-n-that, though I notice it has the
user running Lib/test/regrtest.py directly (the "old" way) instead of
using Tools/scripts/run_tests.py. The latter execs Python as its last
order of business, thus losing the parallel setup of coverage.py.

So, now I understand what's going on. Thanks for your help.
Coverage.py is a great tool.

Skip



More information about the testing-in-python mailing list