[TIP] Resetting or Restarting coverage.py trace
Shai Cantor
shai at sealights.io
Mon Oct 17 08:31:00 PDT 2016
Yup, that's what I'm doing right now.
I've enabled the WTW feature and manually changing context upon a new test
or time frame.
The thing is, I cannot clear the contexts data, when I try to do that it
never records anymore.
I can only change context and keep the old.
Any ideas?
On Mon, Oct 17, 2016 at 6:25 PM, <testing-in-python-request at lists.idyll.org>
wrote:
> Send testing-in-python mailing list submissions to
> testing-in-python at lists.idyll.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.idyll.org/listinfo/testing-in-python
> or, via email, send a message with subject or body 'help' to
> testing-in-python-request at lists.idyll.org
>
> You can reach the person managing the list at
> testing-in-python-owner at lists.idyll.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
>
> Today's Topics:
>
> 1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>
>
> ---------- Forwarded message ----------
> From: Ned Batchelder <ned at nedbatchelder.com>
> To: testing-in-python at lists.idyll.org
> Cc:
> Date: Mon, 17 Oct 2016 11:25:08 -0400
> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>
> Another way would be to somehow get the "who tests what" feature
> implemented, so that you could segregate your results by which test covered
> the code. https://bitbucket.org/ned/coveragepy/issues/170/show-
> who-tests-what
>
> But that is a big effort.
>
> --Ned.
> On 10/17/16 11:17 AM, Shai Cantor wrote:
>
> I wish.
> In some setups, companies spin up servers that are dedicated for testing
> by multiple clients.
> Those cannot be frequently restarted.
> Do you think there can be another way?
>
> On Mon, Oct 17, 2016 at 4:00 PM, <testing-in-python-request@
> lists.idyll.org> wrote:
>
>> Send testing-in-python mailing list submissions to
>> testing-in-python at lists.idyll.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.idyll.org/listinfo/testing-in-python
>> or, via email, send a message with subject or body 'help' to
>> testing-in-python-request at lists.idyll.org
>>
>> You can reach the person managing the list at
>> testing-in-python-owner at lists.idyll.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of testing-in-python digest..."
>>
>> Today's Topics:
>>
>> 1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>> 2. Re: pytest (James)
>>
>>
>> ---------- Forwarded message ----------
>> From: Ned Batchelder <ned at nedbatchelder.com>
>> To: testing-in-python at lists.idyll.org
>> Cc:
>> Date: Mon, 17 Oct 2016 07:09:33 -0400
>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>
>> Wouldn't it be easiest to restart the process between the two phases?
>>
>> --Ned.
>>
>> On 10/17/16 4:22 AM, Shai Cantor wrote:
>>
>> I'm trying to find the coverage of multiple processes that are running on
>> multiple test phases.
>> For example:
>>
>> - A CI server that is conducting tests that make HTTP calls to
>> another server for functional tests phase.
>> - The same CI server is conducting tests that make other HTTP calls
>> to the same server as part of the performance test phase.
>>
>> I would like to measure coverage for the functional test phase, reset the
>> coverage and start measuring again from scratch for the performance test
>> phase.
>>
>> On Sun, Oct 16, 2016 at 10:00 PM, <testing-in-python-request at lis
>> ts.idyll.org> wrote:
>>
>>> Send testing-in-python mailing list submissions to
>>> testing-in-python at lists.idyll.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> http://lists.idyll.org/listinfo/testing-in-python
>>> or, via email, send a message with subject or body 'help' to
>>> testing-in-python-request at lists.idyll.org
>>>
>>> You can reach the person managing the list at
>>> testing-in-python-owner at lists.idyll.org
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of testing-in-python digest..."
>>>
>>> Today's Topics:
>>>
>>> 1. Resetting or Restarting coverage.py trace (Shai Cantor)
>>> 2. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>>> 3. Re: are the absolute paths in .coverage necessary? (Chris Withers)
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Shai Cantor <shai at sealights.io>
>>> To: testing-in-python at lists.idyll.org
>>> Cc:
>>> Date: Sun, 16 Oct 2016 09:36:42 +0300
>>> Subject: [TIP] Resetting or Restarting coverage.py trace
>>> Hi,
>>>
>>> So I'm using coverage.py API
>>>
>>> import coverage
>>> cov = coverage.Coverage()cov.start()
>>> # .. call your code ..
>>> cov.stop()cov.save()
>>>
>>>
>>> The main thread creates a coverage instance and starts it.
>>> During the lifetime of the program I have multiple threads and I need
>>> every once in while to reset or restart the coverage process (or clear
>>> coverage data) from one of the underlying threads.
>>> Problem is that tracing is stopped after I do that. I guess it's because
>>> I start it from another thread.
>>>
>>> Is there a way I can reset/restart coverage from another thread or
>>> manually clear coverage data?
>>>
>>> Thanks
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Ned Batchelder <ned at nedbatchelder.com>
>>> To: testing-in-python at lists.idyll.org
>>> Cc:
>>> Date: Sun, 16 Oct 2016 12:10:48 -0400
>>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>> On 10/16/16 2:36 AM, Shai Cantor wrote:
>>>
>>> Hi,
>>>
>>> So I'm using coverage.py API
>>>
>>> import coverage
>>> cov = coverage.Coverage()cov.start()
>>> # .. call your code ..
>>> cov.stop()cov.save()
>>>
>>>
>>> The main thread creates a coverage instance and starts it.
>>> During the lifetime of the program I have multiple threads and I need
>>> every once in while to reset or restart the coverage process (or clear
>>> coverage data) from one of the underlying threads.
>>> Problem is that tracing is stopped after I do that. I guess it's because
>>> I start it from another thread.
>>>
>>> Is there a way I can reset/restart coverage from another thread or
>>> manually clear coverage data?
>>>
>>> I haven't heard this need before, to occasionally reset or clear the
>>> coverage data from one of many threads. Can you say more about what the
>>> bigger picture is here? There might be an easier way to get it done.
>>>
>>> --Ned.
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Chris Withers <chris at simplistix.co.uk>
>>> To: Ned Batchelder <ned at nedbatchelder.com>, "
>>> testing-in-python at lists.idyll.org" <testing-in-python at lists.idyll.org>
>>> Cc:
>>> Date: Sun, 16 Oct 2016 19:44:38 +0100
>>> Subject: Re: [TIP] are the absolute paths in .coverage necessary?
>>> Hi Ned,
>>>
>>> Sorry, I meant: do the paths recorded in the .coverage files need to be
>>> absolute or could Coverage be changed to just record the path relatively to
>>> the current working directory at the start of execution?
>>>
>>> cheers,
>>>
>>> Chris
>>>
>>>
>>> On 12/10/2016 22:31, Ned Batchelder wrote:
>>>
>>>> Chris, as far as I can tell, they do not have to be absolute. As you can
>>>> see from the sample of my rc file, you can replace an arbitrary prefix
>>>> with *, and it will work just fine. How about this:
>>>>
>>>> [paths]
>>>> mylib =
>>>> mylib
>>>> */workspace/PYTHON/*/mylib
>>>> tests =
>>>> tests
>>>> */workspace/PYTHON/*/tests
>>>>
>>>> --Ned.
>>>>
>>>>
>>>> On 10/12/16 10:59 AM, Chris Withers wrote:
>>>>
>>>>> What I currently have is:
>>>>>
>>>>> [paths]
>>>>> mylib =
>>>>> mylib
>>>>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/mylib
>>>>> tests =
>>>>> tests
>>>>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/tests
>>>>>
>>>>> How can I remove the absolute bits of that?
>>>>>
>>>>> But, back to my original question: why are the paths in .coverage
>>>>> absolute in the first place? :-)
>>>>>
>>>>> Chris
>>>>>
>>>>> On 12/10/2016 15:53, Ned Batchelder wrote:
>>>>>
>>>>>> Chris, can you show the absolute paths you are using? I have this in
>>>>>> my
>>>>>> rc file to deal with varieties of CI:
>>>>>>
>>>>>> [paths]
>>>>>> source =
>>>>>> .
>>>>>> *\coverage\trunk
>>>>>> */coverage/trunk
>>>>>> *\coveragepy
>>>>>>
>>>>>> --Ned.
>>>>>>
>>>>>>
>>>>>> On 10/12/16 10:00 AM, Chris Withers wrote:
>>>>>>
>>>>>>> Hi Ned,
>>>>>>>
>>>>>>> Are the absolute paths necessary in .coverage?
>>>>>>>
>>>>>>> The reason I ask is that if, for example, they were relative to the
>>>>>>> cwd, then I wouldn't need a .coveragerc with a whole bunch of paths
>>>>>>> predicated on the exact filesystem layout of my CI server.
>>>>>>>
>>>>>>> thoughts?
>>>>>>>
>>>>>>> Chris
>>>>>>>
>>>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>>
>>>
>>
>>
>> _______________________________________________
>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>
>> ---------- Forwarded message ---------- From: James <bjlockie at lockie.ca>
>> To: Bruno Oliveira <nicoddemus at gmail.com>, testing-in-python at lists.idyll.
>> org Cc: Date: Mon, 17 Oct 2016 09:00:39 -0400 Subject: Re: [TIP] pytest
>>
>> The output was: a = <testscript.TestAclasss instance at 0x7f3ca6bf83b0>
>> def test_foo(a): > assert a == '1' E assert
>> <testscript.TestAclasss instance at 0x7f3ca6bf83b0> == '1' testscript.py:6:
>> AssertionError 1 failed in 0.03 seconds
>>
>> The missing "self" fixed it.
>> On 2016-10-16 12:56 AM, Bruno Oliveira wrote:
>>
>> Hi James,
>>
>> What error do you get? From your example it is missing a self argument,
>> otherwise it should work just fine.
>>
>> Cheers, Bruno.
>>
>> On Sat, Oct 15, 2016 at 11:03 AM James <bjlockie at lockie.ca> wrote:
>>
>>> I tried to make it work with a class:
>>>
>>> import pytest # testscript.py class TestAclasss:
>>>
>>> def test_foo(a): assert a == '1'
>>> Do I need to do something in the __init__ function?
>>> On 2016-10-14 02:50 PM, Bruno Oliveira wrote:
>>>
>>> Hi James,
>>>
>>> Thanks for posting the complete example.
>>>
>>> There are some problems here, both general Python problems and some
>>> related to how you are using pytest:
>>>
>>> 1. testscript.py has no access to the A function because it doesn’t
>>> import it.
>>> 2. You probably want to declare your fixture in a conftest.py file
>>> so pytest can make it available for all tests in your test suite.
>>> 3. You have to first declare your option using the pytest_addoption
>>> hook.
>>> 4. Since you intend to provide the value passed in the command line
>>> to the tests using a fixture, it is simpler to skip the test in the fixture
>>> itself.
>>>
>>> Here’s the full example with my suggestions:
>>>
>>> # conftest.pyimport pytest
>>> @pytest.fixture(scope='module')def a(request):
>>> value = request.config.getoption("A")
>>> if not value:
>>> pytest.skip('test needs -A option to run')
>>> return value
>>> def pytest_addoption(parser):
>>> parser.addoption("--A", action="store", default=None, help="a option")
>>> # testscript.pydef test_foo(a):
>>> assert a == '1'
>>>
>>> Running it:
>>>
>>> $ py.test testscript.py -q
>>> s
>>> 1 skipped in 0.01 seconds
>>>
>>> $ py.test testscript.py -q --A 1
>>> .
>>> 1 passed in 0.01 seconds
>>>
>>> $ py.test testscript.py -q --A 2
>>> F
>>> ================================== FAILURES ===================================
>>> __________________________________ test_foo ___________________________________
>>>
>>> a = '2'
>>>
>>> def test_foo(a):
>>> > assert a == '1'
>>> E assert '2' == '1'
>>> E - 2
>>> E + 1
>>>
>>> testscript.py:6: AssertionError
>>> 1 failed in 0.07 seconds
>>>
>>> Hope that helps, Bruno.
>>>
>>> On Fri, Oct 14, 2016 at 3:07 PM James <bjlockie at lockie.ca> wrote:
>>>
>>> Here is the exact code: $ cat testconfig.py import pytest
>>> @pytest.fixture(scope='module') def A(request): return
>>> request.config.getoption("--A")
>>> $ cat testscript.py import pytest
>>> skip_A = pytest.mark.skipif( A(pytest.config.request),
>>> reason="need --A option to run" )
>>> $ py.test testscript.py ============================= test session
>>> starts ============================== platform linux2 -- Python 2.7.12,
>>> pytest-2.8.7, py-1.4.31, pluggy-0.3.1 rootdir: /home/rjl/pytest, inifile:
>>> collected 0 items / 1 errors ====================================
>>> ERRORS ==================================== ________________________
>>> ERROR collecting testscript.py ________________________ testscript.py:4: in
>>> <module> A(pytest.config.request),
>>> E NameError: name 'A' is not defined
>>> =========================== 1 error in 0.01 seconds
>>> ============================ Instead of doing pytest.config.getoption
>>> more than one like this: skip_A = pytest.mark.skipif( pytest.config.
>>> getoption("--A"), reason="need --A option to run" ) I want to do it
>>> once in the config file. I might want to use the value of the argument
>>> --A elsewhere so I don't want to do getoption many times.
>>> On 2016-10-14 01:16 PM, Bruno Oliveira wrote:
>>>
>>> Hi,
>>>
>>> After fixing the missing “pytest” import, I get this error:
>>>
>>> foo.py:8: in <module>
>>> A(pytest.config.request),
>>> E AttributeError: 'Config' object has no attribute 'request'
>>>
>>> Please make sure to post a complete example and state clearly what your
>>> problem is and what you are trying to accomplish.
>>>
>>> Cheers, Bruno.
>>>
>>> On Fri, Oct 14, 2016 at 2:03 PM James <bjlockie at lockie.ca> wrote:
>>>
>>> I have this in my config file: @pytest.fixture(scope='module') def
>>> A(request): return request.config.getoption("--A") skip_A =
>>> pytest.mark.skipif( A(pytest.config.request), reason="need --A
>>> option to run" ) but when I run it, it says E NameError: name 'A' is not
>>> defined
>>> _______________________________________________ testing-in-python
>>> mailing list testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>>
>>> _______________________________________________ testing-in-python
>> mailing list testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>
> _______________________________________________
> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20161017/44df1a88/attachment-0001.htm>
More information about the testing-in-python
mailing list