[TIP] Test added (and executed) isn't increasing apparent coverage

[Skip Montanaro]

I'm using Ned Batchelder's coverage package to investigate spots in my code
which aren't being exercised. Looking at the ,cover file for one module, I
saw:

> def encode_oparg(tup):
>     "smash tuple back into oparg int"
!     oparg = 0
!     for elt in tup:
!         oparg = oparg << 8 | elt
!     return oparg


Recognizing that my code doesn't normally use that function, I added an
explicit example to my unit tests:

from rattlesnake import instructions, opcodes, util
...
    def test_util_encode(self):
        self.assertEqual(util.encode_oparg((1, 24, 2)), 71682)
        self.assertEqual(util.encode_oparg(()), 0)


(Actually, the first assert was already there. I only added the second.)
That changed nothing, which surprised me.

I've got a simple Makefile to drive the process:

COVERAGE = $(HOME)/.local/bin/coverage
RATSRC = $(PWD)/Lib/rattlesnake
all : FORCE
        $(COVERAGE) erase
        $(COVERAGE) run -a --source=$(RATSRC) $(HOME)/tmp/junk.py
        $(COVERAGE) run -a --source=$(RATSRC) ./Tools/scripts/run_tests.py
-v test_rattlesnake
        $(COVERAGE) annotate
        $(COVERAGE) report

FORCE :


When I run make, this section of output corresponds to running the unit
tests:

/home/skip/.local/bin/coverage run -a
--source=/home/skip/src/python/cpython/Lib/rattlesnake
./Tools/scripts/run_tests.py -v test_rattlesnake
/home/skip/src/python/cpython/python -u -W default -bb -E -m test -r -w -j
0 -u all,-largefile,-audio,-gui -v test_rattlesnake
== CPython 3.9.0a5+ (heads/register:25d05303a7, Apr 2 2020, 06:27:03) [GCC
9.2.1 20191008]
== Linux-5.3.0-42-generic-x86_64-with-glibc2.30 little-endian
== cwd: /home/skip/src/python/cpython/build/test_python_14692
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Using random seed 7332863
0:00:00 load avg: 5.25 Run tests in parallel using 10 child processes
0:00:00 load avg: 5.25 [1/1] test_rattlesnake passed
test_long_block_function (test.test_rattlesnake.InstructionTest) ... ok
test_nop (test.test_rattlesnake.InstructionTest) ... ok
test_simple_branch_function (test.test_rattlesnake.InstructionTest) ... ok
test_src_dst (test.test_rattlesnake.InstructionTest) ... ok
test_trivial_function (test.test_rattlesnake.InstructionTest) ... ok
test_util_LineNumberDict (test.test_rattlesnake.InstructionTest) ... ok
test_util_decode (test.test_rattlesnake.InstructionTest) ... ok
test_util_encode (test.test_rattlesnake.InstructionTest) ... ok


Here's the output of the report command:

Name                              Stmts   Miss  Cover
-----------------------------------------------------
Lib/rattlesnake/__init__.py           0      0   100%
Lib/rattlesnake/blocks.py            63      4    94%
Lib/rattlesnake/converter.py        271     11    96%
Lib/rattlesnake/decorators.py        42     42     0%
Lib/rattlesnake/instructions.py     145      3    98%
Lib/rattlesnake/opcodes.py          163      0   100%
Lib/rattlesnake/util.py              38      7    82%
-----------------------------------------------------
TOTAL                               722     67    91%


OTOH, if I extract the calls to decode_oparg and encode_oparg into my
junk.py script:

print(decode_oparg(0) == (0,))
print(decode_oparg(71682, False) == (0, 1, 24, 2))
print(encode_oparg((1, 24, 2)) == 71682)
print(encode_oparg(()) == 0)


reported coverage of .../util.py goes up significantly:

Name                              Stmts   Miss  Cover
-----------------------------------------------------
Lib/rattlesnake/__init__.py           0      0   100%
Lib/rattlesnake/blocks.py            63      4    94%
Lib/rattlesnake/converter.py        271     11    96%
Lib/rattlesnake/decorators.py        42     42     0%
Lib/rattlesnake/instructions.py     145      3    98%
Lib/rattlesnake/opcodes.py          163      0   100%
Lib/rattlesnake/util.py              38      2    95%
-----------------------------------------------------
TOTAL                               722     62    91%


Note the execution of test_util_decode and test_util_encode. I think the
unit test run should contribute to the overall coverage, but at least some
of it appears not to for some reason. Is there something I'm doing wrong in
setting up or executing my coverage commands?

Thx,

Skip Montanaro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20200402/adf12172/attachment.html>