[TIP] coverage.py against a server
Ned Batchelder
ned at nedbatchelder.com
Fri Sep 20 09:57:15 PDT 2013
On 9/19/13 3:58 AM, Christoph Buchner wrote:
> re this:
>> Right, --source trumps --include. You've already told coverage that the only interesting files are in ../src, so there's nothing else to include. I should make a warning for inconsistent options like this (ticket: https://bitbucket.org/ned/coveragepy/issue/265/when-using-source-include-is-silently)
> are you sure this is 100% correct? I think there' more nuances here. I have observed during working with my remote-executing script (mail convo up until a couple days ago), that there's a difference in behaviour between only source, and both source and include being specified.
Christoph, reading the coverage.py code, it seems clear to me that when
--source is specified, --include is ignored. I could be mis-reading the
code, though. Can you re-run your tests, and show the results? The
results you describe for the source-but-no-include case seems wrong: it
isn't even limiting coverage to the --source directory.
--Ned.
> Specifically, using this script to run coverage:
>
> #!/usr/bin/env python
> import os
> import sys
> import inspect
> import coverage
> import subprocess
>
> arguments = ''
> if len(sys.argv) > 1:
> arguments = ' '.join(sys.argv[1:])
>
> testdir = os.path.abspath(os.path.dirname(inspect.getfile(inspect.currentframe())))
> os.environ['COVERAGE_PROCESS_START'] = os.path.join(testdir, '.coveragerc')
> os.environ['COVERAGE_FILE'] = os.path.join(testdir, '.coverage')
> cov = coverage.coverage(source=os.path.join(testdir, '..'),
> include=os.path.join(testdir, '..', 'ofStateManager.py'))
>
> cov.erase()
> subprocess.call('coverage run -m py.test ' + arguments, shell=True, cwd=testdir)
> cov.combine()
> cov.html_report(directory=os.path.join(testdir, 'htmlcov'))
> cov.report(show_missing=False)
>
> If I run this as-is, the report contains only ofStateManager.py, as desired.
> If I strip out the include=... part in coverage.coverage(...), I get a coverage report on ofStateManager.py, the tests I run, and a load of things in ~/.local/lib/python2.7/site-packages/ and /usr/share/pyshared/, which I don't want. This was the reason to add that include= option in the first place.
> (fwiw, the whole thing lives at https://github.com/bilderbuchi/ofStateManager/tree/tests currently)
>
> best,
> christoph
>
>
> Ned Batchelder schrieb am 19.09.2013 01:43:
>
>> On 9/18/13 9:10 AM, Lucian Ciufudean wrote:
>>> Sorry for all these iterations, here is a more consistent (but long)
>>> email.
>>>
>> Iterations are fine as long as they bring more detail! Thanks for
>> putting in all the work.
>>
>>> After not being able to run 'coverage run pyc_file.pyc', I created a
>>> dummy driver main.py file that uses the compiled modules.
>>>
>>> root at spahire pyc]# ls -la
>>> total 20
>>> drwxr-xr-x. 2 root root 4096 Sep 18 13:20 .
>>> drwxr-xr-x. 4 root root 4096 Sep 17 15:58 ..
>>> -rw-r--r--. 1 root root 28 Sep 18 13:01 main.py
>>> -rw-------. 1 root root 187 Sep 17 15:59 test_coverage_callee.pyc
>>> -rw-------. 1 root root 212 Sep 17 15:59 test_coverage_caller.pyc
>>> [root at spahire pyc]# cat main.py
>>> import test_coverage_caller
>>>
>>> The source files for the 2 pyc files are here:
>>>
>>> [root at spahire pyc]# ls ../src
>>> test_coverage_callee.py test_coverage_caller.py
>>>
>>> Further, I run coverage, I get a warning but a data file .coverage is
>>> created:
>>>
>>> [root at spahire pyc]# coverage run --source=../src main.py
>>> 1
>>> 2
>>> 2.1
>>> 2.2
>>> 2.3
>>> Coverage.py warning: No data was collected.
>>> [root at spahire pyc]# ls -l .coverage
>>> -rw-r--r--. 1 root root 180 Sep 18 13:23 .coverage
>>>
>> The --source option tells coverage that the only files of interest are
>> the ones in ../src. You never execute any files in ../src, so coverage
>> hasn't collected any data.
>>
>>> --include does not make a difference:
>>>
>>> [root at spahire pyc]# coverage run --source=../src --include='*' main.py
>>> 1
>>> 2
>>> 2.1
>>> 2.2
>>> 2.3
>>> Coverage.py warning: No data was collected.
>>> [root at spahire pyc]# ls -l .coverage
>>> -rw-r--r--. 1 root root 180 Sep 18 13:38 .coverage
>>>
>> Right, --source trumps --include. You've already told coverage that the
>> only interesting files are in ../src, so there's nothing else to
>> include. I should make a warning for inconsistent options like this
>> (ticket:
>> https://bitbucket.org/ned/coveragepy/issue/265/when-using-source-include-is-silently)
>>
>>> As expected, report shows nothing is covered:
>>>
>>> [root at spahire pyc]# coverage report --include='*test*'
>>> Name Stmts Miss Cover
>>> ----------------------------------------------------------------------------
>>> /root/lucian/coverage/module1/src/test_coverage_callee 3 3
>>> 0%
>>> /root/lucian/coverage/module1/src/test_coverage_caller 3 3
>>> 0%
>>> ----------------------------------------------------------------------------
>>> TOTAL 6 6 0%
>>>
>>> report does not accept --source, might this be the root to all evil?
>>>
>> Hmm, that seems like an oversight: I should make that possible (ticket:
>> https://bitbucket.org/ned/coveragepy/issue/266/report-command-doesnt-accept-source-option)
>>> [root at spahire pyc]# coverage report --include='*test*' --source=../src
>>> no such option: --source
>>> Use 'coverage help' for help.
>>>
>>> No warning when using run without options, but the report is again wrong:
>>>
>>> [root at spahire pyc]# coverage run main.py
>>> 1
>>> 2
>>> 2.1
>>> 2.2
>>> 2.3
>>> [root at spahire pyc]# coverage report
>>> Name Stmts Miss Cover
>>> ------------------------------------------
>>> main 1 0 100%
>>> test_coverage_callee NoSource: No source for code:
>>> '/root/lucian/coverage/module1/pyc/test_coverage_callee.py'
>>> test_coverage_caller NoSource: No source for code:
>>> '/root/lucian/coverage/module1/pyc/test_coverage_caller.py'
>>>
>> The problem is that you haven't told coverage how to find the source
>> files that correspond to your .pyc files. The --source option doesn't
>> do that.
>>
>>> Same thing with a configuration file:
>>> [root at spahire pyc]# cat .coveragerc
>>> [run]
>>> source=../src/
>>> include=*
>>>
>> Right, same options, specified in a different way.
>>> So now I turn to the api + the same .coveragerc, the same thing:
>>>
>>> [root at spahire pyc]# cat api.py
>>> import coverage
>>> cov = coverage.coverage()
>>> cov.start()
>>> import test_coverage_caller
>>> cov.stop()
>>> cov.save()
>>>
>>> [root at spahire pyc]# python api.py
>>> 1
>>> 2
>>> 2.1
>>> 2.2
>>> 2.3
>>> Coverage.py warning: No data was collected.
>>>
>> Another run with the same (non-)options.
>>
>>> As for your suggestion with [paths], the docs suggest to me that this
>>> is for combining data.
>> Yes, it is used when combining data, I should have fleshed out my idea
>> more fully. Try creating a .coveragerc file like this:
>>
>> [run]
>> parallel = true
>>
>> [paths]
>> mysources =
>> ../src
>> .
>>
>> Then use "coverage run main.py", then "coverage combine", then "coverage
>> report". If that works, we can talk about how to make it a bit easier.
>>
>> Hope that helps,
>>
>> --Ned.
>>
>>> Lucian
>>>
>>>
>>>
>>> On Wed, Sep 18, 2013 at 1:20 PM, Ned Batchelder <ned at nedbatchelder.com
>>> <mailto:ned at nedbatchelder.com>> wrote:
>>>
>>> On 9/18/13 3:42 AM, Lucian Ciufudean wrote:
>>>> Here is how I ran coverage:
>>>>
>>>> coverage run --source=../src main.py
>>>>
>>>> main.py imports a pyc file that resides in the same folder. The
>>>> source of this pyc file is available in ../src.
>>>> (I created main.py just to go around the bug that you submitted
>>>> above)
>>>>
>>> You don't mention the exact errors you saw (details matter), but
>>> in your previous message you mentioned the problem being while
>>> reporting. You'll also have to specify the source directory
>>> during the "coverage html" command (or coverage.html() call). A
>>> good way to do this is with a .coveragerc file.
>>>
>>> --Ned.
>>>
>>>
>>>> Lucian
>>>>
>>>>
>>>> On Wed, Sep 18, 2013 at 5:05 AM, Ned Batchelder
>>>> <ned at nedbatchelder.com <mailto:ned at nedbatchelder.com>> wrote:
>>>>
>>>> On 9/17/13 7:42 AM, Lucian Ciufudean wrote:
>>>>> Hi guys,
>>>>>
>>>>> I embarked on the road of obtaining a coverage report for
>>>>> functional tests against a server process. The server is
>>>>> deployed as .pyc files, and the source files can be made
>>>>> available in a separate subversion working folder. I
>>>>> wouldn't want to edit any existing .py files.
>>>>>
>>>>> Can coverage work with .pyc files - I am getting errors when
>>>>> using coverage run main.pyc from the command line, so maybe
>>>>> with the API?
>>>> Hmm, you're right: "coverage run foo.pyc" does not work. I've
>>>> created a ticket for this:
>>>> https://bitbucket.org/ned/coveragepy/issue/264/coverage-wont-run-pyc-files
>>>>
>>>>
>>>>
>>>>> I tired this also, my custom code is run but it can not find
>>>>> the source files at the time of coverage.html_report()
>>>>> although I passed the directory of source files to
>>>>> coverage.coverage.
>>>> You'll have to show details of how you tried to run
>>>> coverage. If the .py files are in the places reported by
>>>> your program, then it should work. You can also use the
>>>> [paths] section of a .coveragerc to instruct coverage where
>>>> the files are.
>>>>
>>>> --Ned.
>>>>
>>>>> Thanks a lot,
>>>>> Lucian
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> testing-in-python mailing list
>>>>> testing-in-python at lists.idyll.org
>>>>> <mailto:testing-in-python at lists.idyll.org>
>>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
More information about the testing-in-python
mailing list