[TIP] Resetting or Restarting coverage.py trace

Shai Cantor shai at sealights.io
Mon Oct 17 08:17:57 PDT 2016


I wish.
In some setups, companies spin up servers that are dedicated for testing by
multiple clients.
Those cannot be frequently restarted.
Do you think there can be another way?

On Mon, Oct 17, 2016 at 4:00 PM, <testing-in-python-request at lists.idyll.org>
wrote:

> Send testing-in-python mailing list submissions to
>         testing-in-python at lists.idyll.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://lists.idyll.org/listinfo/testing-in-python
> or, via email, send a message with subject or body 'help' to
>         testing-in-python-request at lists.idyll.org
>
> You can reach the person managing the list at
>         testing-in-python-owner at lists.idyll.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
>
> Today's Topics:
>
>    1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>    2. Re: pytest (James)
>
>
> ---------- Forwarded message ----------
> From: Ned Batchelder <ned at nedbatchelder.com>
> To: testing-in-python at lists.idyll.org
> Cc:
> Date: Mon, 17 Oct 2016 07:09:33 -0400
> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>
> Wouldn't it be easiest to restart the process between the two phases?
>
> --Ned.
>
> On 10/17/16 4:22 AM, Shai Cantor wrote:
>
> I'm trying to find the coverage of multiple processes that are running on
> multiple test phases.
> For example:
>
>    - A CI server that is conducting tests that make HTTP calls to another
>    server for functional tests phase.
>    - The same CI server is conducting tests that make other HTTP calls to
>    the same server as part of the performance test phase.
>
> I would like to measure coverage for the functional test phase, reset the
> coverage and start measuring again from scratch for the performance test
> phase.
>
> On Sun, Oct 16, 2016 at 10:00 PM, <testing-in-python-request@
> lists.idyll.org> wrote:
>
>> Send testing-in-python mailing list submissions to
>>         testing-in-python at lists.idyll.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         http://lists.idyll.org/listinfo/testing-in-python
>> or, via email, send a message with subject or body 'help' to
>>         testing-in-python-request at lists.idyll.org
>>
>> You can reach the person managing the list at
>>         testing-in-python-owner at lists.idyll.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of testing-in-python digest..."
>>
>> Today's Topics:
>>
>>    1. Resetting or Restarting coverage.py trace (Shai Cantor)
>>    2. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>>    3. Re: are the absolute paths in .coverage necessary? (Chris Withers)
>>
>>
>> ---------- Forwarded message ----------
>> From: Shai Cantor <shai at sealights.io>
>> To: testing-in-python at lists.idyll.org
>> Cc:
>> Date: Sun, 16 Oct 2016 09:36:42 +0300
>> Subject: [TIP] Resetting or Restarting coverage.py trace
>> Hi,
>>
>> So I'm using coverage.py API
>>
>> import coverage
>> cov = coverage.Coverage()cov.start()
>> # .. call your code ..
>> cov.stop()cov.save()
>>
>>
>> The main thread creates a coverage instance and starts it.
>> During the lifetime of the program I have multiple threads and I need
>> every once in while to reset or restart the coverage process (or clear
>> coverage data) from one of the underlying threads.
>> Problem is that tracing is stopped after I do that. I guess it's because
>> I start it from another thread.
>>
>> Is there a way I can reset/restart coverage from another thread or
>> manually clear coverage data?
>>
>> Thanks
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Ned Batchelder <ned at nedbatchelder.com>
>> To: testing-in-python at lists.idyll.org
>> Cc:
>> Date: Sun, 16 Oct 2016 12:10:48 -0400
>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>> On 10/16/16 2:36 AM, Shai Cantor wrote:
>>
>> Hi,
>>
>> So I'm using coverage.py API
>>
>> import coverage
>> cov = coverage.Coverage()cov.start()
>> # .. call your code ..
>> cov.stop()cov.save()
>>
>>
>> The main thread creates a coverage instance and starts it.
>> During the lifetime of the program I have multiple threads and I need
>> every once in while to reset or restart the coverage process (or clear
>> coverage data) from one of the underlying threads.
>> Problem is that tracing is stopped after I do that. I guess it's because
>> I start it from another thread.
>>
>> Is there a way I can reset/restart coverage from another thread or
>> manually clear coverage data?
>>
>> I haven't heard this need before, to occasionally reset or clear the
>> coverage data from one of many threads. Can you say more about what the
>> bigger picture is here? There might be an easier way to get it done.
>>
>> --Ned.
>>
>>
>> ---------- Forwarded message ----------
>> From: Chris Withers <chris at simplistix.co.uk>
>> To: Ned Batchelder <ned at nedbatchelder.com>, "testing-in-python at lists.
>> idyll.org" <testing-in-python at lists.idyll.org>
>> Cc:
>> Date: Sun, 16 Oct 2016 19:44:38 +0100
>> Subject: Re: [TIP] are the absolute paths in .coverage necessary?
>> Hi Ned,
>>
>> Sorry, I meant: do the paths recorded in the .coverage files need to be
>> absolute or could Coverage be changed to just record the path relatively to
>> the current working directory at the start of execution?
>>
>> cheers,
>>
>> Chris
>>
>>
>> On 12/10/2016 22:31, Ned Batchelder wrote:
>>
>>> Chris, as far as I can tell, they do not have to be absolute. As you can
>>> see from the sample of my rc file, you can replace an arbitrary prefix
>>> with *, and it will work just fine.  How about this:
>>>
>>> [paths]
>>> mylib =
>>>    mylib
>>>    */workspace/PYTHON/*/mylib
>>> tests =
>>>    tests
>>>    */workspace/PYTHON/*/tests
>>>
>>> --Ned.
>>>
>>>
>>> On 10/12/16 10:59 AM, Chris Withers wrote:
>>>
>>>> What I currently have is:
>>>>
>>>> [paths]
>>>> mylib =
>>>>    mylib
>>>>    /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/mylib
>>>> tests =
>>>>    tests
>>>>    /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/tests
>>>>
>>>> How can I remove the absolute bits of that?
>>>>
>>>> But, back to my original question: why are the paths in .coverage
>>>> absolute in the first place? :-)
>>>>
>>>> Chris
>>>>
>>>> On 12/10/2016 15:53, Ned Batchelder wrote:
>>>>
>>>>> Chris, can you show the absolute paths you are using?  I have this in
>>>>> my
>>>>> rc file to deal with varieties of CI:
>>>>>
>>>>>      [paths]
>>>>>      source =
>>>>>          .
>>>>>          *\coverage\trunk
>>>>>          */coverage/trunk
>>>>>          *\coveragepy
>>>>>
>>>>> --Ned.
>>>>>
>>>>>
>>>>> On 10/12/16 10:00 AM, Chris Withers wrote:
>>>>>
>>>>>> Hi Ned,
>>>>>>
>>>>>> Are the absolute paths necessary in .coverage?
>>>>>>
>>>>>> The reason I ask is that if, for example, they were relative to the
>>>>>> cwd, then I wouldn't need a .coveragerc with a whole bunch of paths
>>>>>> predicated on the exact filesystem layout of my CI server.
>>>>>>
>>>>>> thoughts?
>>>>>>
>>>>>> Chris
>>>>>>
>>>>>
>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>>
>
>
> _______________________________________________
> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>
>
>
>
> ---------- Forwarded message ----------
> From: James <bjlockie at lockie.ca>
> To: Bruno Oliveira <nicoddemus at gmail.com>,
> testing-in-python at lists.idyll.org
> Cc:
> Date: Mon, 17 Oct 2016 09:00:39 -0400
> Subject: Re: [TIP] pytest
>
> The output was:
> a = <testscript.TestAclasss instance at 0x7f3ca6bf83b0>
>
>     def test_foo(a):
> >       assert a == '1'
> E       assert <testscript.TestAclasss instance at 0x7f3ca6bf83b0> == '1'
>
> testscript.py:6: AssertionError
> 1 failed in 0.03 seconds
>
>
> The missing "self" fixed it.
>
> On 2016-10-16 12:56 AM, Bruno Oliveira wrote:
>
> Hi James,
>
> What error do you get? From your example it is missing a self argument,
> otherwise it should work just fine.
>
> Cheers,
> Bruno.
>>
> On Sat, Oct 15, 2016 at 11:03 AM James <bjlockie at lockie.ca> wrote:
>
>> I tried to make it work with a class:
>>
>> import pytest
>>
>> # testscript.py
>> class TestAclasss:
>>
>>
>>     def test_foo(a):
>>         assert a == '1'
>> Do I need to do something in the __init__ function?
>>
>>
>>
>>
>> On 2016-10-14 02:50 PM, Bruno Oliveira wrote:
>>
>> Hi James,
>>
>> Thanks for posting the complete example.
>>
>> There are some problems here, both general Python problems and some
>> related to how you are using pytest:
>>
>>    1. testscript.py has no access to the A function because it doesn’t
>>    import it.
>>    2. You probably want to declare your fixture in a conftest.py file so
>>    pytest can make it available for all tests in your test suite.
>>    3. You have to first declare your option using the pytest_addoption
>>    hook.
>>    4. Since you intend to provide the value passed in the command line
>>    to the tests using a fixture, it is simpler to skip the test in the fixture
>>    itself.
>>
>> Here’s the full example with my suggestions:
>>
>> # conftest.pyimport pytest
>> @pytest.fixture(scope='module')def a(request):
>>     value = request.config.getoption("A")
>>     if not value:
>>         pytest.skip('test needs -A option to run')
>>     return value
>> def pytest_addoption(parser):
>>     parser.addoption("--A", action="store", default=None, help="a option")
>> # testscript.pydef test_foo(a):
>>     assert a == '1'
>>
>> Running it:
>>
>> $ py.test testscript.py -q
>> s
>> 1 skipped in 0.01 seconds
>>
>> $ py.test testscript.py -q --A 1
>> .
>> 1 passed in 0.01 seconds
>>
>> $ py.test testscript.py -q --A 2
>> F
>> ================================== FAILURES ===================================
>> __________________________________ test_foo ___________________________________
>>
>> a = '2'
>>
>>     def test_foo(a):
>> >       assert a == '1'
>> E       assert '2' == '1'
>> E         - 2
>> E         + 1
>>
>> testscript.py:6: AssertionError
>> 1 failed in 0.07 seconds
>>
>> Hope that helps,
>> Bruno.
>>>>
>>
>> On Fri, Oct 14, 2016 at 3:07 PM James <bjlockie at lockie.ca> wrote:
>>
>>
>> Here is the exact code:
>> $ cat testconfig.py
>> import pytest
>>
>>
>> @pytest.fixture(scope='module')
>> def A(request):
>>     return request.config.getoption("--A")
>>
>> $ cat testscript.py
>> import pytest
>>
>>
>> skip_A = pytest.mark.skipif(
>>     A(pytest.config.request),
>>     reason="need --A option to run"
>> )
>>
>>
>> $ py.test testscript.py
>> ============================= test session starts
>> ==============================
>> platform linux2 -- Python 2.7.12, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
>> rootdir: /home/rjl/pytest, inifile:
>> collected 0 items / 1 errors
>>
>> ==================================== ERRORS
>> ====================================
>> ________________________ ERROR collecting testscript.py
>> ________________________
>> testscript.py:4: in <module>
>>     A(pytest.config.request),
>>
>> E   NameError: name 'A' is not defined
>> =========================== 1 error in 0.01 seconds
>> ============================
>>
>> Instead of doing
>> pytest.config.getoption more than one like this:
>> skip_A = pytest.mark.skipif( pytest.config.getoption("--A"), reason="need
>> --A option to run" )
>> I want to do it once in the config file.
>> I might want to use the value of the argument --A elsewhere so I don't
>> want to do getoption many times.
>>
>>
>> On 2016-10-14 01:16 PM, Bruno Oliveira wrote:
>>
>> Hi,
>>
>> After fixing the missing “pytest” import, I get this error:
>>
>> foo.py:8: in <module>
>>     A(pytest.config.request),
>> E   AttributeError: 'Config' object has no attribute 'request'
>>
>> Please make sure to post a complete example and state clearly what your
>> problem is and what you are trying to accomplish.
>>
>> Cheers,
>> Bruno.
>>>>
>> On Fri, Oct 14, 2016 at 2:03 PM James <bjlockie at lockie.ca> wrote:
>>
>> I have this in my config file:
>> @pytest.fixture(scope='module')
>> def A(request):
>>     return request.config.getoption("--A")
>>
>> skip_A = pytest.mark.skipif(
>>     A(pytest.config.request),
>>     reason="need --A option to run"
>> )
>>
>> but when I run it, it says
>> E   NameError: name 'A' is not defined
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>>
>>
>>
>>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20161017/32f12658/attachment-0001.htm>


More information about the testing-in-python mailing list