[TIP] Resetting or Restarting coverage.py trace
Ned Batchelder
ned at nedbatchelder.com
Mon Oct 17 08:25:08 PDT 2016
Another way would be to somehow get the "who tests what" feature
implemented, so that you could segregate your results by which test
covered the code.
https://bitbucket.org/ned/coveragepy/issues/170/show-who-tests-what
But that is a big effort.
--Ned.
On 10/17/16 11:17 AM, Shai Cantor wrote:
> I wish.
> In some setups, companies spin up servers that are dedicated for
> testing by multiple clients.
> Those cannot be frequently restarted.
> Do you think there can be another way?
>
> On Mon, Oct 17, 2016 at 4:00 PM,
> <testing-in-python-request at lists.idyll.org
> <mailto:testing-in-python-request at lists.idyll.org>> wrote:
>
> Send testing-in-python mailing list submissions to
> testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.idyll.org/listinfo/testing-in-python
> <http://lists.idyll.org/listinfo/testing-in-python>
> or, via email, send a message with subject or body 'help' to
> testing-in-python-request at lists.idyll.org
> <mailto:testing-in-python-request at lists.idyll.org>
>
> You can reach the person managing the list at
> testing-in-python-owner at lists.idyll.org
> <mailto:testing-in-python-owner at lists.idyll.org>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
>
> Today's Topics:
>
> 1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
> 2. Re: pytest (James)
>
>
> ---------- Forwarded message ----------
> From: Ned Batchelder <ned at nedbatchelder.com
> <mailto:ned at nedbatchelder.com>>
> To: testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org>
> Cc:
> Date: Mon, 17 Oct 2016 07:09:33 -0400
> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>
> Wouldn't it be easiest to restart the process between the two phases?
>
> --Ned.
>
>
> On 10/17/16 4:22 AM, Shai Cantor wrote:
>> I'm trying to find the coverage of multiple processes that are
>> running on multiple test phases.
>> For example:
>>
>> * A CI server that is conducting tests that make HTTP calls to
>> another server for functional tests phase.
>> * The same CI server is conducting tests that make other HTTP
>> calls to the same server as part of the performance test phase.
>>
>> I would like to measure coverage for the functional test phase,
>> reset the coverage and start measuring again from scratch for the
>> performance test phase.
>>
>> On Sun, Oct 16, 2016 at 10:00 PM,
>> <testing-in-python-request at lists.idyll.org
>> <mailto:testing-in-python-request at lists.idyll.org>> wrote:
>>
>> Send testing-in-python mailing list submissions to
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.idyll.org/listinfo/testing-in-python
>> <http://lists.idyll.org/listinfo/testing-in-python>
>> or, via email, send a message with subject or body 'help' to
>> testing-in-python-request at lists.idyll.org
>> <mailto:testing-in-python-request at lists.idyll.org>
>>
>> You can reach the person managing the list at
>> testing-in-python-owner at lists.idyll.org
>> <mailto:testing-in-python-owner at lists.idyll.org>
>>
>> When replying, please edit your Subject line so it is more
>> specific
>> than "Re: Contents of testing-in-python digest..."
>>
>> Today's Topics:
>>
>> 1. Resetting or Restarting coverage.py trace (Shai Cantor)
>> 2. Re: Resetting or Restarting coverage.py trace (Ned
>> Batchelder)
>> 3. Re: are the absolute paths in .coverage necessary?
>> (Chris Withers)
>>
>>
>> ---------- Forwarded message ----------
>> From: Shai Cantor <shai at sealights.io <mailto:shai at sealights.io>>
>> To: testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> Cc:
>> Date: Sun, 16 Oct 2016 09:36:42 +0300
>> Subject: [TIP] Resetting or Restarting coverage.py trace
>> Hi,
>>
>> So I'm using coverage.py API
>>
>> import coverage
>>
>> cov = coverage.Coverage()
>> cov.start()
>>
>> # .. call your code ..
>>
>> cov.stop()
>> cov.save()
>>
>>
>> The main thread creates a coverage instance and starts it.
>> During the lifetime of the program I have multiple threads
>> and I need every once in while to reset or restart the
>> coverage process (or clear coverage data) from one of the
>> underlying threads.
>> Problem is that tracing is stopped after I do that. I guess
>> it's because I start it from another thread.
>>
>> Is there a way I can reset/restart coverage from another
>> thread or manually clear coverage data?
>>
>> Thanks
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Ned Batchelder <ned at nedbatchelder.com
>> <mailto:ned at nedbatchelder.com>>
>> To: testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> Cc:
>> Date: Sun, 16 Oct 2016 12:10:48 -0400
>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>> On 10/16/16 2:36 AM, Shai Cantor wrote:
>>> Hi,
>>>
>>> So I'm using coverage.py API
>>> import coverage
>>>
>>> cov = coverage.Coverage()
>>> cov.start()
>>>
>>> # .. call your code ..
>>>
>>> cov.stop()
>>> cov.save()
>>>
>>> The main thread creates a coverage instance and starts it.
>>> During the lifetime of the program I have multiple threads
>>> and I need every once in while to reset or restart the
>>> coverage process (or clear coverage data) from one of the
>>> underlying threads.
>>> Problem is that tracing is stopped after I do that. I guess
>>> it's because I start it from another thread.
>>>
>>> Is there a way I can reset/restart coverage from another
>>> thread or manually clear coverage data?
>>>
>> I haven't heard this need before, to occasionally reset or
>> clear the coverage data from one of many threads. Can you say
>> more about what the bigger picture is here? There might be an
>> easier way to get it done.
>>
>> --Ned.
>>
>>
>> ---------- Forwarded message ----------
>> From: Chris Withers <chris at simplistix.co.uk
>> <mailto:chris at simplistix.co.uk>>
>> To: Ned Batchelder <ned at nedbatchelder.com
>> <mailto:ned at nedbatchelder.com>>,
>> "testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>"
>> <testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>>
>> Cc:
>> Date: Sun, 16 Oct 2016 19:44:38 +0100
>> Subject: Re: [TIP] are the absolute paths in .coverage necessary?
>> Hi Ned,
>>
>> Sorry, I meant: do the paths recorded in the .coverage files
>> need to be absolute or could Coverage be changed to just
>> record the path relatively to the current working directory
>> at the start of execution?
>>
>> cheers,
>>
>> Chris
>>
>>
>> On 12/10/2016 22:31, Ned Batchelder wrote:
>>
>> Chris, as far as I can tell, they do not have to be
>> absolute. As you can
>> see from the sample of my rc file, you can replace an
>> arbitrary prefix
>> with *, and it will work just fine. How about this:
>>
>> [paths]
>> mylib =
>> mylib
>> */workspace/PYTHON/*/mylib
>> tests =
>> tests
>> */workspace/PYTHON/*/tests
>>
>> --Ned.
>>
>>
>> On 10/12/16 10:59 AM, Chris Withers wrote:
>>
>> What I currently have is:
>>
>> [paths]
>> mylib =
>> mylib
>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/mylib
>> tests =
>> tests
>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/tests
>>
>> How can I remove the absolute bits of that?
>>
>> But, back to my original question: why are the paths
>> in .coverage
>> absolute in the first place? :-)
>>
>> Chris
>>
>> On 12/10/2016 15:53, Ned Batchelder wrote:
>>
>> Chris, can you show the absolute paths you are
>> using? I have this in my
>> rc file to deal with varieties of CI:
>>
>> [paths]
>> source =
>> .
>> *\coverage\trunk
>> */coverage/trunk
>> *\coveragepy
>>
>> --Ned.
>>
>>
>> On 10/12/16 10:00 AM, Chris Withers wrote:
>>
>> Hi Ned,
>>
>> Are the absolute paths necessary in .coverage?
>>
>> The reason I ask is that if, for example,
>> they were relative to the
>> cwd, then I wouldn't need a .coveragerc with
>> a whole bunch of paths
>> predicated on the exact filesystem layout of
>> my CI server.
>>
>> thoughts?
>>
>> Chris
>>
>>
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> http://lists.idyll.org/listinfo/testing-in-python
>> <http://lists.idyll.org/listinfo/testing-in-python>
>>
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> http://lists.idyll.org/listinfo/testing-in-python
>> <http://lists.idyll.org/listinfo/testing-in-python>
> ---------- Forwarded message ---------- From: James
> <bjlockie at lockie.ca <mailto:bjlockie at lockie.ca>> To: Bruno
> Oliveira <nicoddemus at gmail.com <mailto:nicoddemus at gmail.com>>,
> testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org> Cc: Date: Mon, 17 Oct
> 2016 09:00:39 -0400 Subject: Re: [TIP] pytest
>
> The output was: a = <testscript.TestAclasss instance at
> 0x7f3ca6bf83b0> def test_foo(a): > assert a == '1'
> E assert <testscript.TestAclasss instance at 0x7f3ca6bf83b0>
> == '1' testscript.py:6: AssertionError 1 failed in 0.03 seconds
>
> The missing "self" fixed it.
>
> On 2016-10-16 12:56 AM, Bruno Oliveira wrote:
>>
>> Hi James,
>>
>> What error do you get? From your example it is missing a |self|
>> argument, otherwise it should work just fine.
>>
>> Cheers, Bruno.
>>
>>
>> On Sat, Oct 15, 2016 at 11:03 AM James <bjlockie at lockie.ca
>> <mailto:bjlockie at lockie.ca>> wrote:
>>
>> I tried to make it work with a class:
>>
>> import pytest # testscript.py class TestAclasss:
>>
>> def test_foo(a): assert a == '1'
>>
>> Do I need to do something in the __init__ function?
>> On 2016-10-14 02:50 PM, Bruno Oliveira wrote:
>>>
>>> Hi James,
>>>
>>> Thanks for posting the complete example.
>>>
>>> There are some problems here, both general Python problems
>>> and some related to how you are using pytest:
>>>
>>> 1. |testscript.py| has no access to the |A| function
>>> because it doesn’t import it.
>>> 2. You probably want to declare your fixture in a
>>> |conftest.py| file so pytest can make it available for
>>> all tests in your test suite.
>>> 3. You have to first declare your option using the
>>> |pytest_addoption| hook.
>>> 4. Since you intend to provide the value passed in the
>>> command line to the tests using a fixture, it is simpler
>>> to skip the test in the fixture itself.
>>>
>>> Here’s the full example with my suggestions:
>>>
>>> |# conftest.py import pytest @pytest.fixture(scope='module')
>>> def a(request): value = request.config.getoption("A") if not
>>> value: pytest.skip('test needs -A option to run') return
>>> value def pytest_addoption(parser): parser.addoption("--A",
>>> action="store", default=None, help="a option") #
>>> testscript.py def test_foo(a): assert a == '1' |
>>>
>>> Running it:
>>>
>>> |$ py.test testscript.py -q s 1 skipped in 0.01 seconds $
>>> py.test testscript.py -q --A 1 . 1 passed in 0.01 seconds $
>>> py.test testscript.py -q --A 2 F
>>> ================================== FAILURES
>>> ===================================
>>> __________________________________ test_foo
>>> ___________________________________ a = '2' def test_foo(a):
>>> > assert a == '1' E assert '2' == '1' E - 2 E + 1
>>> testscript.py:6: AssertionError 1 failed in 0.07 seconds |
>>>
>>> Hope that helps, Bruno.
>>>
>>>
>>> On Fri, Oct 14, 2016 at 3:07 PM James <bjlockie at lockie.ca
>>> <mailto:bjlockie at lockie.ca>> wrote:
>>>
>>> Here is the exact code: $ cat testconfig.py import pytest
>>> @pytest.fixture(scope='module') def A(request):
>>> return request.config.getoption("--A")
>>> $ cat testscript.py import pytest
>>> skip_A = pytest.mark.skipif(
>>> A(pytest.config.request), reason="need --A option to
>>> run" )
>>> $ py.test testscript.py =============================
>>> test session starts ==============================
>>> platform linux2 -- Python 2.7.12, pytest-2.8.7,
>>> py-1.4.31, pluggy-0.3.1 rootdir: /home/rjl/pytest,
>>> inifile: collected 0 items / 1 errors
>>> ==================================== ERRORS
>>> ====================================
>>> ________________________ ERROR collecting testscript.py
>>> ________________________ testscript.py:4: in <module>
>>> A(pytest.config.request),
>>> E NameError: name 'A' is not defined
>>> =========================== 1 error in 0.01 seconds
>>> ============================ Instead of doing
>>> pytest.config.getoptionmore than one like this: skip_A =
>>> pytest.mark.skipif( pytest.config.getoption("--A"),
>>> reason="need --A option to run" ) I want to do it once
>>> in the config file. I might want to use the value of the
>>> argument --A elsewhere so I don't want to do getoption
>>> many times.
>>> On 2016-10-14 01:16 PM, Bruno Oliveira wrote:
>>>>
>>>> Hi,
>>>>
>>>> After fixing the missing “pytest” import, I get this error:
>>>>
>>>> |foo.py:8: in <module> A(pytest.config.request), E
>>>> AttributeError: 'Config' object has no attribute
>>>> 'request' |
>>>>
>>>> Please make sure to post a complete example and state
>>>> clearly what your problem is and what you are trying to
>>>> accomplish.
>>>>
>>>> Cheers, Bruno.
>>>>
>>>>
>>>> On Fri, Oct 14, 2016 at 2:03 PM James
>>>> <bjlockie at lockie.ca <mailto:bjlockie at lockie.ca>> wrote:
>>>>
>>>> I have this in my config file:
>>>> @pytest.fixture(scope='module') def A(request):
>>>> return request.config.getoption("--A") skip_A =
>>>> pytest.mark.skipif( A(pytest.config.request),
>>>> reason="need --A option to run" ) but when I
>>>> run it, it says E NameError: name 'A' is not
>>>> defined
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> <mailto:testing-in-python at lists.idyll.org>
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>> <http://lists.idyll.org/listinfo/testing-in-python>
>>>>
> _______________________________________________ testing-in-python
> mailing list testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org>
> http://lists.idyll.org/listinfo/testing-in-python
> <http://lists.idyll.org/listinfo/testing-in-python>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20161017/cb5da00d/attachment-0001.htm>
More information about the testing-in-python
mailing list