[TIP] coverage.py against a server

Christoph Buchner bilderbuchi at phononoia.at
Thu Sep 19 00:58:13 PDT 2013


re this: 
> Right, --source trumps --include. You've already told coverage that the only interesting files are in ../src, so there's nothing else to include. I should make a warning for inconsistent options like this (ticket: https://bitbucket.org/ned/coveragepy/issue/265/when-using-source-include-is-silently)

are you sure this is 100% correct? I think there' more nuances here. I have observed during working with my remote-executing script (mail convo up until a couple days ago), that there's a difference in behaviour between only source, and both source and include being specified.
Specifically, using this script to run coverage:

    #!/usr/bin/env python
    import os
    import sys
    import inspect
    import coverage
    import subprocess

    arguments = ''
    if len(sys.argv) > 1:
	    arguments = ' '.join(sys.argv[1:])

    testdir = os.path.abspath(os.path.dirname(inspect.getfile(inspect.currentframe())))
    os.environ['COVERAGE_PROCESS_START'] = os.path.join(testdir, '.coveragerc')
    os.environ['COVERAGE_FILE'] = os.path.join(testdir, '.coverage')
    cov = coverage.coverage(source=os.path.join(testdir, '..'),
					    include=os.path.join(testdir, '..', 'ofStateManager.py'))

    cov.erase()
    subprocess.call('coverage run -m py.test ' + arguments, shell=True, cwd=testdir)
    cov.combine()
    cov.html_report(directory=os.path.join(testdir, 'htmlcov'))
    cov.report(show_missing=False)

If I run this as-is, the report contains only ofStateManager.py, as desired.
If I strip out the include=... part in coverage.coverage(...), I get a coverage report on ofStateManager.py, the tests I run, and a load of things in ~/.local/lib/python2.7/site-packages/ and /usr/share/pyshared/, which I don't want. This was the reason to add that include= option in the first place.
(fwiw, the whole thing lives at https://github.com/bilderbuchi/ofStateManager/tree/tests currently)

best,
christoph


Ned Batchelder schrieb am 19.09.2013 01:43:

> On 9/18/13 9:10 AM, Lucian Ciufudean wrote:
>> Sorry for all these iterations, here is a more consistent (but long) 
>> email.
>>
> Iterations are fine as long as they bring more detail!  Thanks for 
> putting in all the work.
> 
>> After not being able to run 'coverage run pyc_file.pyc', I created a 
>> dummy driver main.py file that uses the compiled modules.
>>
>> root at spahire pyc]# ls -la
>> total 20
>> drwxr-xr-x. 2 root root 4096 Sep 18 13:20 .
>> drwxr-xr-x. 4 root root 4096 Sep 17 15:58 ..
>> -rw-r--r--. 1 root root   28 Sep 18 13:01 main.py
>> -rw-------. 1 root root  187 Sep 17 15:59 test_coverage_callee.pyc
>> -rw-------. 1 root root  212 Sep 17 15:59 test_coverage_caller.pyc
>> [root at spahire pyc]# cat  main.py
>> import test_coverage_caller
>>
>> The source files for the 2 pyc files are here:
>>
>> [root at spahire pyc]# ls ../src
>> test_coverage_callee.py  test_coverage_caller.py
>>
>> Further, I run coverage, I get a warning but a data file .coverage is 
>> created:
>>
>> [root at spahire pyc]# coverage run --source=../src main.py
>> 1
>> 2
>> 2.1
>> 2.2
>> 2.3
>> Coverage.py warning: No data was collected.
>> [root at spahire pyc]# ls -l .coverage
>> -rw-r--r--. 1 root root 180 Sep 18 13:23 .coverage
>>
> The --source option tells coverage that the only files of interest are 
> the ones in ../src.  You never execute any files in ../src, so coverage 
> hasn't collected any data.
> 
>> --include does not make a difference:
>>
>> [root at spahire pyc]# coverage run --source=../src --include='*' main.py
>> 1
>> 2
>> 2.1
>> 2.2
>> 2.3
>> Coverage.py warning: No data was collected.
>> [root at spahire pyc]# ls -l .coverage
>> -rw-r--r--. 1 root root 180 Sep 18 13:38 .coverage
>>
> Right, --source trumps --include.  You've already told coverage that the 
> only interesting files are in ../src, so there's nothing else to 
> include.  I should make a warning for inconsistent options like this 
> (ticket: 
> https://bitbucket.org/ned/coveragepy/issue/265/when-using-source-include-is-silently)
> 
>> As expected, report shows nothing is covered:
>>
>> [root at spahire pyc]# coverage report --include='*test*'
>> Name   Stmts   Miss  Cover
>> ----------------------------------------------------------------------------
>> /root/lucian/coverage/module1/src/test_coverage_callee       3      3 
>>     0%
>> /root/lucian/coverage/module1/src/test_coverage_caller       3      3 
>>     0%
>> ----------------------------------------------------------------------------
>> TOTAL      6      6     0%
>>
>> report does not accept --source, might this be the root to all evil?
>>
> Hmm, that seems like an oversight: I should make that possible (ticket: 
> https://bitbucket.org/ned/coveragepy/issue/266/report-command-doesnt-accept-source-option)
>> [root at spahire pyc]# coverage report --include='*test*' --source=../src
>> no such option: --source
>> Use 'coverage help' for help.
>>
>> No warning when using run without options, but the report is again wrong:
>>
>> [root at spahire pyc]# coverage run main.py
>> 1
>> 2
>> 2.1
>> 2.2
>> 2.3
>> [root at spahire pyc]# coverage report
>> Name                   Stmts   Miss  Cover
>> ------------------------------------------
>> main                       1      0   100%
>> test_coverage_callee   NoSource: No source for code: 
>> '/root/lucian/coverage/module1/pyc/test_coverage_callee.py'
>> test_coverage_caller   NoSource: No source for code: 
>> '/root/lucian/coverage/module1/pyc/test_coverage_caller.py'
>>
> The problem is that you haven't told coverage how to find the source 
> files that correspond to your .pyc files.  The --source option doesn't 
> do that.
> 
>> Same thing with a configuration file:
>> [root at spahire pyc]# cat .coveragerc
>> [run]
>> source=../src/
>> include=*
>>
> Right, same options, specified in a different way.
>> So now I turn to the api + the same .coveragerc, the same thing:
>>
>> [root at spahire pyc]# cat api.py
>> import coverage
>> cov = coverage.coverage()
>> cov.start()
>> import test_coverage_caller
>> cov.stop()
>> cov.save()
>>
>> [root at spahire pyc]# python api.py
>> 1
>> 2
>> 2.1
>> 2.2
>> 2.3
>> Coverage.py warning: No data was collected.
>>
> Another run with the same (non-)options.
> 
>> As for your suggestion with [paths], the docs suggest to me that this 
>> is for combining data.
> Yes, it is used when combining data, I should have fleshed out my idea 
> more fully.  Try creating a .coveragerc file like this:
> 
>     [run]
>     parallel = true
> 
>     [paths]
>     mysources =
>         ../src
>         .
> 
> Then use "coverage run main.py", then "coverage combine", then "coverage 
> report".  If that works, we can talk about how to make it a bit easier.
> 
> Hope that helps,
> 
> --Ned.
> 
>>
>> Lucian
>>
>>
>>
>> On Wed, Sep 18, 2013 at 1:20 PM, Ned Batchelder <ned at nedbatchelder.com 
>> <mailto:ned at nedbatchelder.com>> wrote:
>>
>>     On 9/18/13 3:42 AM, Lucian Ciufudean wrote:
>>>     Here is how I ran coverage:
>>>
>>>         coverage run --source=../src main.py
>>>
>>>     main.py imports a pyc file that resides in the same folder. The
>>>     source of this pyc file is available in ../src.
>>>     (I created main.py just to go around the bug that you submitted
>>>     above)
>>>
>>     You don't mention the exact errors you saw (details matter), but
>>     in your previous message you mentioned the problem being while
>>     reporting.  You'll also have to specify the source directory
>>     during the "coverage html" command (or coverage.html() call).  A
>>     good way to do this is with a .coveragerc file.
>>
>>     --Ned.
>>
>>
>>>     Lucian
>>>
>>>
>>>     On Wed, Sep 18, 2013 at 5:05 AM, Ned Batchelder
>>>     <ned at nedbatchelder.com <mailto:ned at nedbatchelder.com>> wrote:
>>>
>>>         On 9/17/13 7:42 AM, Lucian Ciufudean wrote:
>>>>         Hi guys,
>>>>
>>>>         I embarked on the road of obtaining a coverage report for
>>>>         functional tests against a server process. The server is
>>>>         deployed as .pyc files, and the source files can be made
>>>>         available in a separate subversion working folder. I
>>>>         wouldn't want to edit any existing .py files.
>>>>
>>>>         Can coverage work with .pyc files - I am getting errors when
>>>>         using coverage run main.pyc from the command line, so maybe
>>>>         with the API?
>>>
>>>         Hmm, you're right: "coverage run foo.pyc" does not work. I've
>>>         created a ticket for this:
>>>         https://bitbucket.org/ned/coveragepy/issue/264/coverage-wont-run-pyc-files
>>>
>>>
>>>
>>>>         I tired this also, my custom code is run but it can not find
>>>>         the source files at the time of coverage.html_report()
>>>>         although I passed the directory of source files to
>>>>         coverage.coverage.
>>>
>>>         You'll have to show details of how you tried to run
>>>         coverage.  If the .py files are in the places reported by
>>>         your program, then it should work.  You can also use the
>>>         [paths] section of a .coveragerc to instruct coverage where
>>>         the files are.
>>>
>>>         --Ned.
>>>
>>>>
>>>>         Thanks a lot,
>>>>         Lucian
>>>>
>>>>
>>>>         _______________________________________________
>>>>         testing-in-python mailing list
>>>>         testing-in-python at lists.idyll.org 
>>>>         <mailto:testing-in-python at lists.idyll.org>
>>>>         http://lists.idyll.org/listinfo/testing-in-python
>>>
>>>
>>
>>
> 
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
> 




More information about the testing-in-python mailing list