[TIP] Resetting or Restarting coverage.py trace
Ned Batchelder
ned at nedbatchelder.com
Tue Oct 18 03:33:47 PDT 2016
Just to be clear: there is a bit of hack code in coverage.py now, with
"WTW" as a moniker, but it is not a feature, it is a hack to test out
some theories about how a future feature might work.
--Ned.
On 10/17/16 11:31 AM, Shai Cantor wrote:
> Yup, that's what I'm doing right now.
> I've enabled the WTW feature and manually changing context upon a new
> test or time frame.
> The thing is, I cannot clear the contexts data, when I try to do that
> it never records anymore.
> I can only change context and keep the old.
>
> Any ideas?
>
> On Mon, Oct 17, 2016 at 6:25 PM,
> <testing-in-python-request at lists.idyll.org
> <mailto:testing-in-python-request at lists.idyll.org>> wrote:
>
> Send testing-in-python mailing list submissions to
> testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org>
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.idyll.org/listinfo/testing-in-python
> <http://lists.idyll.org/listinfo/testing-in-python>
> or, via email, send a message with subject or body 'help' to
> testing-in-python-request at lists.idyll.org
> <mailto:testing-in-python-request at lists.idyll.org>
>
> You can reach the person managing the list at
> testing-in-python-owner at lists.idyll.org
> <mailto:testing-in-python-owner at lists.idyll.org>
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
>
> Today's Topics:
>
> 1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>
>
> ---------- Forwarded message ----------
> From: Ned Batchelder <ned at nedbatchelder.com
> <mailto:ned at nedbatchelder.com>>
> To: testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org>
> Cc:
> Date: Mon, 17 Oct 2016 11:25:08 -0400
> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>
> Another way would be to somehow get the "who tests what" feature
> implemented, so that you could segregate your results by which
> test covered the code.
> https://bitbucket.org/ned/coveragepy/issues/170/show-who-tests-what
> <https://bitbucket.org/ned/coveragepy/issues/170/show-who-tests-what>
>
> But that is a big effort.
>
> --Ned.
>
> On 10/17/16 11:17 AM, Shai Cantor wrote:
>> I wish.
>> In some setups, companies spin up servers that are dedicated for
>> testing by multiple clients.
>> Those cannot be frequently restarted.
>> Do you think there can be another way?
>>
>> On Mon, Oct 17, 2016 at 4:00 PM,
>> <testing-in-python-request at lists.idyll.org
>> <mailto:testing-in-python-request at lists.idyll.org>> wrote:
>>
>> Send testing-in-python mailing list submissions to
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.idyll.org/listinfo/testing-in-python
>> <http://lists.idyll.org/listinfo/testing-in-python>
>> or, via email, send a message with subject or body 'help' to
>> testing-in-python-request at lists.idyll.org
>> <mailto:testing-in-python-request at lists.idyll.org>
>>
>> You can reach the person managing the list at
>> testing-in-python-owner at lists.idyll.org
>> <mailto:testing-in-python-owner at lists.idyll.org>
>>
>> When replying, please edit your Subject line so it is more
>> specific
>> than "Re: Contents of testing-in-python digest..."
>>
>> Today's Topics:
>>
>> 1. Re: Resetting or Restarting coverage.py trace (Ned
>> Batchelder)
>> 2. Re: pytest (James)
>>
>>
>> ---------- Forwarded message ----------
>> From: Ned Batchelder <ned at nedbatchelder.com
>> <mailto:ned at nedbatchelder.com>>
>> To: testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> Cc:
>> Date: Mon, 17 Oct 2016 07:09:33 -0400
>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>
>> Wouldn't it be easiest to restart the process between the two
>> phases?
>>
>> --Ned.
>>
>>
>> On 10/17/16 4:22 AM, Shai Cantor wrote:
>>> I'm trying to find the coverage of multiple processes that
>>> are running on multiple test phases.
>>> For example:
>>>
>>> * A CI server that is conducting tests that make HTTP
>>> calls to another server for functional tests phase.
>>> * The same CI server is conducting tests that make other
>>> HTTP calls to the same server as part of the performance
>>> test phase.
>>>
>>> I would like to measure coverage for the functional test
>>> phase, reset the coverage and start measuring again from
>>> scratch for the performance test phase.
>>>
>>> On Sun, Oct 16, 2016 at 10:00 PM,
>>> <testing-in-python-request at lists.idyll.org
>>> <mailto:testing-in-python-request at lists.idyll.org>> wrote:
>>>
>>> Send testing-in-python mailing list submissions to
>>> testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>
>>> http://lists.idyll.org/listinfo/testing-in-python
>>> <http://lists.idyll.org/listinfo/testing-in-python>
>>> or, via email, send a message with subject or body 'help' to
>>> testing-in-python-request at lists.idyll.org
>>> <mailto:testing-in-python-request at lists.idyll.org>
>>>
>>> You can reach the person managing the list at
>>> testing-in-python-owner at lists.idyll.org
>>> <mailto:testing-in-python-owner at lists.idyll.org>
>>>
>>> When replying, please edit your Subject line so it is
>>> more specific
>>> than "Re: Contents of testing-in-python digest..."
>>>
>>> Today's Topics:
>>>
>>> 1. Resetting or Restarting coverage.py trace (Shai
>>> Cantor)
>>> 2. Re: Resetting or Restarting coverage.py trace (Ned
>>> Batchelder)
>>> 3. Re: are the absolute paths in .coverage necessary?
>>> (Chris Withers)
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Shai Cantor <shai at sealights.io
>>> <mailto:shai at sealights.io>>
>>> To: testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>
>>> Cc:
>>> Date: Sun, 16 Oct 2016 09:36:42 +0300
>>> Subject: [TIP] Resetting or Restarting coverage.py trace
>>> Hi,
>>>
>>> So I'm using coverage.py API
>>>
>>> import coverage
>>>
>>> cov = coverage.Coverage()
>>> cov.start()
>>>
>>> # .. call your code ..
>>>
>>> cov.stop()
>>> cov.save()
>>>
>>>
>>> The main thread creates a coverage instance and starts it.
>>> During the lifetime of the program I have multiple
>>> threads and I need every once in while to reset or
>>> restart the coverage process (or clear coverage data)
>>> from one of the underlying threads.
>>> Problem is that tracing is stopped after I do that. I
>>> guess it's because I start it from another thread.
>>>
>>> Is there a way I can reset/restart coverage from another
>>> thread or manually clear coverage data?
>>>
>>> Thanks
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Ned Batchelder <ned at nedbatchelder.com
>>> <mailto:ned at nedbatchelder.com>>
>>> To: testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>
>>> Cc:
>>> Date: Sun, 16 Oct 2016 12:10:48 -0400
>>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>> On 10/16/16 2:36 AM, Shai Cantor wrote:
>>>> Hi,
>>>>
>>>> So I'm using coverage.py API
>>>> import coverage
>>>>
>>>> cov = coverage.Coverage()
>>>> cov.start()
>>>>
>>>> # .. call your code ..
>>>>
>>>> cov.stop()
>>>> cov.save()
>>>>
>>>> The main thread creates a coverage instance and starts it.
>>>> During the lifetime of the program I have multiple
>>>> threads and I need every once in while to reset or
>>>> restart the coverage process (or clear coverage data)
>>>> from one of the underlying threads.
>>>> Problem is that tracing is stopped after I do that. I
>>>> guess it's because I start it from another thread.
>>>>
>>>> Is there a way I can reset/restart coverage from
>>>> another thread or manually clear coverage data?
>>>>
>>> I haven't heard this need before, to occasionally reset
>>> or clear the coverage data from one of many threads. Can
>>> you say more about what the bigger picture is here?
>>> There might be an easier way to get it done.
>>>
>>> --Ned.
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Chris Withers <chris at simplistix.co.uk
>>> <mailto:chris at simplistix.co.uk>>
>>> To: Ned Batchelder <ned at nedbatchelder.com
>>> <mailto:ned at nedbatchelder.com>>,
>>> "testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>"
>>> <testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>>
>>> Cc:
>>> Date: Sun, 16 Oct 2016 19:44:38 +0100
>>> Subject: Re: [TIP] are the absolute paths in .coverage
>>> necessary?
>>> Hi Ned,
>>>
>>> Sorry, I meant: do the paths recorded in the .coverage
>>> files need to be absolute or could Coverage be changed
>>> to just record the path relatively to the current
>>> working directory at the start of execution?
>>>
>>> cheers,
>>>
>>> Chris
>>>
>>>
>>> On 12/10/2016 22:31, Ned Batchelder wrote:
>>>
>>> Chris, as far as I can tell, they do not have to be
>>> absolute. As you can
>>> see from the sample of my rc file, you can replace
>>> an arbitrary prefix
>>> with *, and it will work just fine. How about this:
>>>
>>> [paths]
>>> mylib =
>>> mylib
>>> */workspace/PYTHON/*/mylib
>>> tests =
>>> tests
>>> */workspace/PYTHON/*/tests
>>>
>>> --Ned.
>>>
>>>
>>> On 10/12/16 10:59 AM, Chris Withers wrote:
>>>
>>> What I currently have is:
>>>
>>> [paths]
>>> mylib =
>>> mylib
>>>
>>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/mylib
>>> tests =
>>> tests
>>>
>>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/tests
>>>
>>> How can I remove the absolute bits of that?
>>>
>>> But, back to my original question: why are the
>>> paths in .coverage
>>> absolute in the first place? :-)
>>>
>>> Chris
>>>
>>> On 12/10/2016 15:53, Ned Batchelder wrote:
>>>
>>> Chris, can you show the absolute paths you
>>> are using? I have this in my
>>> rc file to deal with varieties of CI:
>>>
>>> [paths]
>>> source =
>>> .
>>> *\coverage\trunk
>>> */coverage/trunk
>>> *\coveragepy
>>>
>>> --Ned.
>>>
>>>
>>> On 10/12/16 10:00 AM, Chris Withers wrote:
>>>
>>> Hi Ned,
>>>
>>> Are the absolute paths necessary in
>>> .coverage?
>>>
>>> The reason I ask is that if, for
>>> example, they were relative to the
>>> cwd, then I wouldn't need a .coveragerc
>>> with a whole bunch of paths
>>> predicated on the exact filesystem
>>> layout of my CI server.
>>>
>>> thoughts?
>>>
>>> Chris
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>
>>> http://lists.idyll.org/listinfo/testing-in-python
>>> <http://lists.idyll.org/listinfo/testing-in-python>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing list
>>> testing-in-python at lists.idyll.org
>>> <mailto:testing-in-python at lists.idyll.org>
>>> http://lists.idyll.org/listinfo/testing-in-python
>>> <http://lists.idyll.org/listinfo/testing-in-python>
>> ---------- Forwarded message ---------- From: James
>> <bjlockie at lockie.ca <mailto:bjlockie at lockie.ca>> To: Bruno
>> Oliveira <nicoddemus at gmail.com
>> <mailto:nicoddemus at gmail.com>>,
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org> Cc: Date: Mon, 17
>> Oct 2016 09:00:39 -0400 Subject: Re: [TIP] pytest
>>
>> The output was: a = <testscript.TestAclasss instance at
>> 0x7f3ca6bf83b0> def test_foo(a): > assert a == '1'
>> E assert <testscript.TestAclasss instance at
>> 0x7f3ca6bf83b0> == '1' testscript.py:6: AssertionError 1
>> failed in 0.03 seconds
>>
>> The missing "self" fixed it.
>>
>> On 2016-10-16 12:56 AM, Bruno Oliveira wrote:
>>>
>>> Hi James,
>>>
>>> What error do you get? From your example it is missing a
>>> |self| argument, otherwise it should work just fine.
>>>
>>> Cheers, Bruno.
>>>
>>>
>>> On Sat, Oct 15, 2016 at 11:03 AM James <bjlockie at lockie.ca
>>> <mailto:bjlockie at lockie.ca>> wrote:
>>>
>>> I tried to make it work with a class:
>>>
>>> import pytest # testscript.py class TestAclasss:
>>>
>>> def test_foo(a): assert a == '1'
>>>
>>> Do I need to do something in the __init__ function?
>>> On 2016-10-14 02:50 PM, Bruno Oliveira wrote:
>>>>
>>>> Hi James,
>>>>
>>>> Thanks for posting the complete example.
>>>>
>>>> There are some problems here, both general Python
>>>> problems and some related to how you are using pytest:
>>>>
>>>> 1. |testscript.py| has no access to the |A| function
>>>> because it doesn’t import it.
>>>> 2. You probably want to declare your fixture in a
>>>> |conftest.py| file so pytest can make it available
>>>> for all tests in your test suite.
>>>> 3. You have to first declare your option using the
>>>> |pytest_addoption| hook.
>>>> 4. Since you intend to provide the value passed in the
>>>> command line to the tests using a fixture, it is
>>>> simpler to skip the test in the fixture itself.
>>>>
>>>> Here’s the full example with my suggestions:
>>>>
>>>> |# conftest.py import pytest
>>>> @pytest.fixture(scope='module') def a(request): value =
>>>> request.config.getoption("A") if not value:
>>>> pytest.skip('test needs -A option to run') return value
>>>> def pytest_addoption(parser): parser.addoption("--A",
>>>> action="store", default=None, help="a option") #
>>>> testscript.py def test_foo(a): assert a == '1' |
>>>>
>>>> Running it:
>>>>
>>>> |$ py.test testscript.py -q s 1 skipped in 0.01 seconds
>>>> $ py.test testscript.py -q --A 1 . 1 passed in 0.01
>>>> seconds $ py.test testscript.py -q --A 2 F
>>>> ================================== FAILURES
>>>> ===================================
>>>> __________________________________ test_foo
>>>> ___________________________________ a = '2' def
>>>> test_foo(a): > assert a == '1' E assert '2' == '1' E -
>>>> 2 E + 1 testscript.py:6: AssertionError 1 failed in
>>>> 0.07 seconds |
>>>>
>>>> Hope that helps, Bruno.
>>>>
>>>>
>>>> On Fri, Oct 14, 2016 at 3:07 PM James
>>>> <bjlockie at lockie.ca <mailto:bjlockie at lockie.ca>> wrote:
>>>>
>>>> Here is the exact code: $ cat testconfig.py import
>>>> pytest
>>>> @pytest.fixture(scope='module') def A(request):
>>>> return request.config.getoption("--A")
>>>> $ cat testscript.py import pytest
>>>> skip_A = pytest.mark.skipif(
>>>> A(pytest.config.request), reason="need --A
>>>> option to run" )
>>>> $ py.test testscript.py
>>>> ============================= test session starts
>>>> ============================== platform linux2 --
>>>> Python 2.7.12, pytest-2.8.7, py-1.4.31,
>>>> pluggy-0.3.1 rootdir: /home/rjl/pytest, inifile:
>>>> collected 0 items / 1 errors
>>>> ==================================== ERRORS
>>>> ====================================
>>>> ________________________ ERROR collecting
>>>> testscript.py ________________________
>>>> testscript.py:4: in <module>
>>>> A(pytest.config.request),
>>>> E NameError: name 'A' is not defined
>>>> =========================== 1 error in 0.01 seconds
>>>> ============================ Instead of doing
>>>> pytest.config.getoptionmore than one like this:
>>>> skip_A = pytest.mark.skipif(
>>>> pytest.config.getoption("--A"), reason="need --A
>>>> option to run" ) I want to do it once in the config
>>>> file. I might want to use the value of the argument
>>>> --A elsewhere so I don't want to do getoption many
>>>> times.
>>>> On 2016-10-14 01:16 PM, Bruno Oliveira wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> After fixing the missing “pytest” import, I get
>>>>> this error:
>>>>>
>>>>> |foo.py:8: in <module> A(pytest.config.request), E
>>>>> AttributeError: 'Config' object has no attribute
>>>>> 'request' |
>>>>>
>>>>> Please make sure to post a complete example and
>>>>> state clearly what your problem is and what you
>>>>> are trying to accomplish.
>>>>>
>>>>> Cheers, Bruno.
>>>>>
>>>>>
>>>>> On Fri, Oct 14, 2016 at 2:03 PM James
>>>>> <bjlockie at lockie.ca <mailto:bjlockie at lockie.ca>>
>>>>> wrote:
>>>>>
>>>>> I have this in my config file:
>>>>> @pytest.fixture(scope='module') def
>>>>> A(request): return
>>>>> request.config.getoption("--A") skip_A =
>>>>> pytest.mark.skipif(
>>>>> A(pytest.config.request), reason="need --A
>>>>> option to run" ) but when I run it, it says
>>>>> E NameError: name 'A' is not defined
>>>>> _______________________________________________
>>>>> testing-in-python mailing list
>>>>> testing-in-python at lists.idyll.org
>>>>> <mailto:testing-in-python at lists.idyll.org>
>>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>> <http://lists.idyll.org/listinfo/testing-in-python>
>>>>>
>>>>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> http://lists.idyll.org/listinfo/testing-in-python
>> <http://lists.idyll.org/listinfo/testing-in-python>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> <mailto:testing-in-python at lists.idyll.org>
>> http://lists.idyll.org/listinfo/testing-in-python
>> <http://lists.idyll.org/listinfo/testing-in-python>
> _______________________________________________ testing-in-python
> mailing list testing-in-python at lists.idyll.org
> <mailto:testing-in-python at lists.idyll.org>
> http://lists.idyll.org/listinfo/testing-in-python
> <http://lists.idyll.org/listinfo/testing-in-python>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20161018/3503919b/attachment-0001.html>
More information about the testing-in-python
mailing list