[TIP] coverage.py against a server
Christoph Buchner
bilderbuchi at phononoia.at
Wed Sep 25 02:27:56 PDT 2013
Ned Batchelder schrieb am 20.09.2013 18:57:
> On 9/19/13 3:58 AM, Christoph Buchner wrote:
>> re this:
>>> Right, --source trumps --include. You've already told coverage that the
>>> only interesting files are in ../src, so there's nothing else to
>>> include. I should make a warning for inconsistent options like this
>>> (ticket:
>>> https://bitbucket.org/ned/coveragepy/issue/265/when-using-source-include-is-silently)
>> are you sure this is 100% correct? I think there' more nuances here. I
>> have observed during working with my remote-executing script (mail convo
>> up until a couple days ago), that there's a difference in behaviour
>> between only source, and both source and include being specified.
>
> Christoph, reading the coverage.py code, it seems clear to me that when
> --source is specified, --include is ignored. I could be mis-reading the
> code, though. Can you re-run your tests, and show the results? The
> results you describe for the source-but-no-include case seems wrong: it
> isn't even limiting coverage to the --source directory.
>
> --Ned.
Ned, sorry for the delay. Using the repo commit https://github.com/bilderbuchi/ofStateManager/tree/5a61f2159fe6d4da6b3a60521e9e62c82b634cae
I get the following log output demonstrating this behaviour:
https://gist.github.com/bilderbuchi/6697184
best,
thanks,
christoph
>
>> Specifically, using this script to run coverage:
>>
>> #!/usr/bin/env python
>> import os
>> import sys
>> import inspect
>> import coverage
>> import subprocess
>>
>> arguments = ''
>> if len(sys.argv) > 1:
>> arguments = ' '.join(sys.argv[1:])
>>
>> testdir =
>> os.path.abspath(os.path.dirname(inspect.getfile(inspect.currentframe())))
>> os.environ['COVERAGE_PROCESS_START'] = os.path.join(testdir,
>> '.coveragerc')
>> os.environ['COVERAGE_FILE'] = os.path.join(testdir, '.coverage')
>> cov = coverage.coverage(source=os.path.join(testdir, '..'),
>> include=os.path.join(testdir, '..', 'ofStateManager.py'))
>>
>> cov.erase()
>> subprocess.call('coverage run -m py.test ' + arguments, shell=True,
>> cwd=testdir)
>> cov.combine()
>> cov.html_report(directory=os.path.join(testdir, 'htmlcov'))
>> cov.report(show_missing=False)
>>
>> If I run this as-is, the report contains only ofStateManager.py, as
>> desired.
>> If I strip out the include=... part in coverage.coverage(...), I get a
>> coverage report on ofStateManager.py, the tests I run, and a load of
>> things in ~/.local/lib/python2.7/site-packages/ and /usr/share/pyshared/,
>> which I don't want. This was the reason to add that include= option in
>> the first place.
>> (fwiw, the whole thing lives at
>> https://github.com/bilderbuchi/ofStateManager/tree/tests currently)
>>
>> best,
>> christoph
>>
>>
>> Ned Batchelder schrieb am 19.09.2013 01:43:
>>
>>> On 9/18/13 9:10 AM, Lucian Ciufudean wrote:
>>>> Sorry for all these iterations, here is a more consistent (but long)
>>>> email.
>>>>
>>> Iterations are fine as long as they bring more detail! Thanks for
>>> putting in all the work.
>>>
>>>> After not being able to run 'coverage run pyc_file.pyc', I created a
>>>> dummy driver main.py file that uses the compiled modules.
>>>>
>>>> root at spahire pyc]# ls -la
>>>> total 20
>>>> drwxr-xr-x. 2 root root 4096 Sep 18 13:20 .
>>>> drwxr-xr-x. 4 root root 4096 Sep 17 15:58 ..
>>>> -rw-r--r--. 1 root root 28 Sep 18 13:01 main.py
>>>> -rw-------. 1 root root 187 Sep 17 15:59 test_coverage_callee.pyc
>>>> -rw-------. 1 root root 212 Sep 17 15:59 test_coverage_caller.pyc
>>>> [root at spahire pyc]# cat main.py
>>>> import test_coverage_caller
>>>>
>>>> The source files for the 2 pyc files are here:
>>>>
>>>> [root at spahire pyc]# ls ../src
>>>> test_coverage_callee.py test_coverage_caller.py
>>>>
>>>> Further, I run coverage, I get a warning but a data file .coverage is
>>>> created:
>>>>
>>>> [root at spahire pyc]# coverage run --source=../src main.py
>>>> 1
>>>> 2
>>>> 2.1
>>>> 2.2
>>>> 2.3
>>>> Coverage.py warning: No data was collected.
>>>> [root at spahire pyc]# ls -l .coverage
>>>> -rw-r--r--. 1 root root 180 Sep 18 13:23 .coverage
>>>>
>>> The --source option tells coverage that the only files of interest are
>>> the ones in ../src. You never execute any files in ../src, so coverage
>>> hasn't collected any data.
>>>
>>>> --include does not make a difference:
>>>>
>>>> [root at spahire pyc]# coverage run --source=../src --include='*' main.py
>>>> 1
>>>> 2
>>>> 2.1
>>>> 2.2
>>>> 2.3
>>>> Coverage.py warning: No data was collected.
>>>> [root at spahire pyc]# ls -l .coverage
>>>> -rw-r--r--. 1 root root 180 Sep 18 13:38 .coverage
>>>>
>>> Right, --source trumps --include. You've already told coverage that the
>>> only interesting files are in ../src, so there's nothing else to
>>> include. I should make a warning for inconsistent options like this
>>> (ticket:
>>> https://bitbucket.org/ned/coveragepy/issue/265/when-using-source-include-is-silently)
>>>
>>>> As expected, report shows nothing is covered:
>>>>
>>>> [root at spahire pyc]# coverage report --include='*test*'
>>>> Name Stmts Miss Cover
>>>> ----------------------------------------------------------------------------
>>>> /root/lucian/coverage/module1/src/test_coverage_callee 3 3
>>>> 0%
>>>> /root/lucian/coverage/module1/src/test_coverage_caller 3 3
>>>> 0%
>>>> ----------------------------------------------------------------------------
>>>> TOTAL 6 6 0%
>>>>
>>>> report does not accept --source, might this be the root to all evil?
>>>>
>>> Hmm, that seems like an oversight: I should make that possible (ticket:
>>> https://bitbucket.org/ned/coveragepy/issue/266/report-command-doesnt-accept-source-option)
>>>> [root at spahire pyc]# coverage report --include='*test*' --source=../src
>>>> no such option: --source
>>>> Use 'coverage help' for help.
>>>>
>>>> No warning when using run without options, but the report is again
>>>> wrong:
>>>>
>>>> [root at spahire pyc]# coverage run main.py
>>>> 1
>>>> 2
>>>> 2.1
>>>> 2.2
>>>> 2.3
>>>> [root at spahire pyc]# coverage report
>>>> Name Stmts Miss Cover
>>>> ------------------------------------------
>>>> main 1 0 100%
>>>> test_coverage_callee NoSource: No source for code:
>>>> '/root/lucian/coverage/module1/pyc/test_coverage_callee.py'
>>>> test_coverage_caller NoSource: No source for code:
>>>> '/root/lucian/coverage/module1/pyc/test_coverage_caller.py'
>>>>
>>> The problem is that you haven't told coverage how to find the source
>>> files that correspond to your .pyc files. The --source option doesn't
>>> do that.
>>>
>>>> Same thing with a configuration file:
>>>> [root at spahire pyc]# cat .coveragerc
>>>> [run]
>>>> source=../src/
>>>> include=*
>>>>
>>> Right, same options, specified in a different way.
>>>> So now I turn to the api + the same .coveragerc, the same thing:
>>>>
>>>> [root at spahire pyc]# cat api.py
>>>> import coverage
>>>> cov = coverage.coverage()
>>>> cov.start()
>>>> import test_coverage_caller
>>>> cov.stop()
>>>> cov.save()
>>>>
>>>> [root at spahire pyc]# python api.py
>>>> 1
>>>> 2
>>>> 2.1
>>>> 2.2
>>>> 2.3
>>>> Coverage.py warning: No data was collected.
>>>>
>>> Another run with the same (non-)options.
>>>
>>>> As for your suggestion with [paths], the docs suggest to me that this
>>>> is for combining data.
>>> Yes, it is used when combining data, I should have fleshed out my idea
>>> more fully. Try creating a .coveragerc file like this:
>>>
>>> [run]
>>> parallel = true
>>>
>>> [paths]
>>> mysources =
>>> ../src
>>> .
>>>
>>> Then use "coverage run main.py", then "coverage combine", then "coverage
>>> report". If that works, we can talk about how to make it a bit easier.
>>>
>>> Hope that helps,
>>>
>>> --Ned.
>>>
>>>> Lucian
>>>>
>>>>
>>>>
>>>> On Wed, Sep 18, 2013 at 1:20 PM, Ned Batchelder <ned at nedbatchelder.com
>>>> <mailto:ned at nedbatchelder.com>> wrote:
>>>>
>>>> On 9/18/13 3:42 AM, Lucian Ciufudean wrote:
>>>>> Here is how I ran coverage:
>>>>>
>>>>> coverage run --source=../src main.py
>>>>>
>>>>> main.py imports a pyc file that resides in the same folder. The
>>>>> source of this pyc file is available in ../src.
>>>>> (I created main.py just to go around the bug that you submitted
>>>>> above)
>>>>>
>>>> You don't mention the exact errors you saw (details matter), but
>>>> in your previous message you mentioned the problem being while
>>>> reporting. You'll also have to specify the source directory
>>>> during the "coverage html" command (or coverage.html() call). A
>>>> good way to do this is with a .coveragerc file.
>>>>
>>>> --Ned.
>>>>
>>>>
>>>>> Lucian
>>>>>
>>>>>
>>>>> On Wed, Sep 18, 2013 at 5:05 AM, Ned Batchelder
>>>>> <ned at nedbatchelder.com <mailto:ned at nedbatchelder.com>> wrote:
>>>>>
>>>>> On 9/17/13 7:42 AM, Lucian Ciufudean wrote:
>>>>>> Hi guys,
>>>>>>
>>>>>> I embarked on the road of obtaining a coverage report for
>>>>>> functional tests against a server process. The server is
>>>>>> deployed as .pyc files, and the source files can be made
>>>>>> available in a separate subversion working folder. I
>>>>>> wouldn't want to edit any existing .py files.
>>>>>>
>>>>>> Can coverage work with .pyc files - I am getting errors when
>>>>>> using coverage run main.pyc from the command line, so maybe
>>>>>> with the API?
>>>>> Hmm, you're right: "coverage run foo.pyc" does not work. I've
>>>>> created a ticket for this:
>>>>> https://bitbucket.org/ned/coveragepy/issue/264/coverage-wont-run-pyc-files
>>>>>
>>>>>
>>>>>
>>>>>> I tired this also, my custom code is run but it can not find
>>>>>> the source files at the time of coverage.html_report()
>>>>>> although I passed the directory of source files to
>>>>>> coverage.coverage.
>>>>> You'll have to show details of how you tried to run
>>>>> coverage. If the .py files are in the places reported by
>>>>> your program, then it should work. You can also use the
>>>>> [paths] section of a .coveragerc to instruct coverage where
>>>>> the files are.
>>>>>
>>>>> --Ned.
>>>>>
>>>>>> Thanks a lot,
>>>>>> Lucian
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> testing-in-python mailing list
>>>>>> testing-in-python at lists.idyll.org
>>>>>> <mailto:testing-in-python at lists.idyll.org>
>>>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>>
>>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>>
>
More information about the testing-in-python
mailing list