[TIP] Resetting or Restarting coverage.py trace
Shai Cantor
shai at sealights.io
Tue Oct 18 06:20:52 PDT 2016
Ned,
I understand it is a hack.
Can we expect to see it officially some time?
Just clarifying for everyone who's interested:
WTW means who-tests-what.
Meaning, coverage per test.
On Tue, Oct 18, 2016 at 1:33 PM, <testing-in-python-request at lists.idyll.org>
wrote:
> Send testing-in-python mailing list submissions to
> testing-in-python at lists.idyll.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.idyll.org/listinfo/testing-in-python
> or, via email, send a message with subject or body 'help' to
> testing-in-python-request at lists.idyll.org
>
> You can reach the person managing the list at
> testing-in-python-owner at lists.idyll.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of testing-in-python digest..."
>
> Today's Topics:
>
> 1. testfixtures 4.12.0 Released! (Chris Withers)
> 2. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>
>
> ---------- Forwarded message ----------
> From: Chris Withers <chris at simplistix.co.uk>
> To: testing-in-python at lists.idyll.org, Python List <python-list at python.org>,
> simplistix at googlegroups.com
> Cc:
> Date: Tue, 18 Oct 2016 09:07:09 +0100
> Subject: [TIP] testfixtures 4.12.0 Released!
> Hi All,
>
> I'm pleased to announce the release of testfixtures 4.12.0 featuring the
> following:
>
> - Add support for specifying a callable to extract rows from log records
> when using LogCapture().
>
> - Add support for recursive comparison of log messages with
> LogCapture().
>
> The package is on PyPI and a full list of all the links to docs, issue
> trackers and the like can be found here:
>
> https://github.com/Simplistix/testfixtures
>
> Any questions, please do ask on the Testing in Python list or on the
> Simplistix open source mailing list...
>
> cheers,
>
> Chris
>
>
>
>
> ---------- Forwarded message ----------
> From: Ned Batchelder <ned at nedbatchelder.com>
> To: testing-in-python at lists.idyll.org
> Cc:
> Date: Tue, 18 Oct 2016 06:33:47 -0400
> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>
> Just to be clear: there is a bit of hack code in coverage.py now, with
> "WTW" as a moniker, but it is not a feature, it is a hack to test out some
> theories about how a future feature might work.
>
> --Ned.
>
> On 10/17/16 11:31 AM, Shai Cantor wrote:
>
> Yup, that's what I'm doing right now.
> I've enabled the WTW feature and manually changing context upon a new test
> or time frame.
> The thing is, I cannot clear the contexts data, when I try to do that it
> never records anymore.
> I can only change context and keep the old.
>
> Any ideas?
>
> On Mon, Oct 17, 2016 at 6:25 PM, <testing-in-python-request@
> lists.idyll.org> wrote:
>
>> Send testing-in-python mailing list submissions to
>> testing-in-python at lists.idyll.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> http://lists.idyll.org/listinfo/testing-in-python
>> or, via email, send a message with subject or body 'help' to
>> testing-in-python-request at lists.idyll.org
>>
>> You can reach the person managing the list at
>> testing-in-python-owner at lists.idyll.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of testing-in-python digest..."
>>
>> Today's Topics:
>>
>> 1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>>
>>
>> ---------- Forwarded message ----------
>> From: Ned Batchelder <ned at nedbatchelder.com>
>> To: testing-in-python at lists.idyll.org
>> Cc:
>> Date: Mon, 17 Oct 2016 11:25:08 -0400
>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>
>> Another way would be to somehow get the "who tests what" feature
>> implemented, so that you could segregate your results by which test covered
>> the code. https://bitbucket.org/ned/coveragepy/issues/170/show-who-
>> tests-what
>>
>> But that is a big effort.
>>
>> --Ned.
>> On 10/17/16 11:17 AM, Shai Cantor wrote:
>>
>> I wish.
>> In some setups, companies spin up servers that are dedicated for testing
>> by multiple clients.
>> Those cannot be frequently restarted.
>> Do you think there can be another way?
>>
>> On Mon, Oct 17, 2016 at 4:00 PM, <testing-in-python-request at lis
>> ts.idyll.org> wrote:
>>
>>> Send testing-in-python mailing list submissions to
>>> testing-in-python at lists.idyll.org
>>>
>>> To subscribe or unsubscribe via the World Wide Web, visit
>>> http://lists.idyll.org/listinfo/testing-in-python
>>> or, via email, send a message with subject or body 'help' to
>>> testing-in-python-request at lists.idyll.org
>>>
>>> You can reach the person managing the list at
>>> testing-in-python-owner at lists.idyll.org
>>>
>>> When replying, please edit your Subject line so it is more specific
>>> than "Re: Contents of testing-in-python digest..."
>>>
>>> Today's Topics:
>>>
>>> 1. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>>> 2. Re: pytest (James)
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Ned Batchelder <ned at nedbatchelder.com>
>>> To: testing-in-python at lists.idyll.org
>>> Cc:
>>> Date: Mon, 17 Oct 2016 07:09:33 -0400
>>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>>
>>> Wouldn't it be easiest to restart the process between the two phases?
>>>
>>> --Ned.
>>>
>>> On 10/17/16 4:22 AM, Shai Cantor wrote:
>>>
>>> I'm trying to find the coverage of multiple processes that are running
>>> on multiple test phases.
>>> For example:
>>>
>>> - A CI server that is conducting tests that make HTTP calls to
>>> another server for functional tests phase.
>>> - The same CI server is conducting tests that make other HTTP calls
>>> to the same server as part of the performance test phase.
>>>
>>> I would like to measure coverage for the functional test phase, reset
>>> the coverage and start measuring again from scratch for the performance
>>> test phase.
>>>
>>> On Sun, Oct 16, 2016 at 10:00 PM, <testing-in-python-request at lis
>>> ts.idyll.org> wrote:
>>>
>>>> Send testing-in-python mailing list submissions to
>>>> testing-in-python at lists.idyll.org
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>> or, via email, send a message with subject or body 'help' to
>>>> testing-in-python-request at lists.idyll.org
>>>>
>>>> You can reach the person managing the list at
>>>> testing-in-python-owner at lists.idyll.org
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of testing-in-python digest..."
>>>>
>>>> Today's Topics:
>>>>
>>>> 1. Resetting or Restarting coverage.py trace (Shai Cantor)
>>>> 2. Re: Resetting or Restarting coverage.py trace (Ned Batchelder)
>>>> 3. Re: are the absolute paths in .coverage necessary? (Chris Withers)
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Shai Cantor <shai at sealights.io>
>>>> To: testing-in-python at lists.idyll.org
>>>> Cc:
>>>> Date: Sun, 16 Oct 2016 09:36:42 +0300
>>>> Subject: [TIP] Resetting or Restarting coverage.py trace
>>>> Hi,
>>>>
>>>> So I'm using coverage.py API
>>>>
>>>> import coverage
>>>> cov = coverage.Coverage()cov.start()
>>>> # .. call your code ..
>>>> cov.stop()cov.save()
>>>>
>>>>
>>>> The main thread creates a coverage instance and starts it.
>>>> During the lifetime of the program I have multiple threads and I need
>>>> every once in while to reset or restart the coverage process (or clear
>>>> coverage data) from one of the underlying threads.
>>>> Problem is that tracing is stopped after I do that. I guess it's
>>>> because I start it from another thread.
>>>>
>>>> Is there a way I can reset/restart coverage from another thread or
>>>> manually clear coverage data?
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Ned Batchelder <ned at nedbatchelder.com>
>>>> To: testing-in-python at lists.idyll.org
>>>> Cc:
>>>> Date: Sun, 16 Oct 2016 12:10:48 -0400
>>>> Subject: Re: [TIP] Resetting or Restarting coverage.py trace
>>>> On 10/16/16 2:36 AM, Shai Cantor wrote:
>>>>
>>>> Hi,
>>>>
>>>> So I'm using coverage.py API
>>>>
>>>> import coverage
>>>> cov = coverage.Coverage()cov.start()
>>>> # .. call your code ..
>>>> cov.stop()cov.save()
>>>>
>>>>
>>>> The main thread creates a coverage instance and starts it.
>>>> During the lifetime of the program I have multiple threads and I need
>>>> every once in while to reset or restart the coverage process (or clear
>>>> coverage data) from one of the underlying threads.
>>>> Problem is that tracing is stopped after I do that. I guess it's
>>>> because I start it from another thread.
>>>>
>>>> Is there a way I can reset/restart coverage from another thread or
>>>> manually clear coverage data?
>>>>
>>>> I haven't heard this need before, to occasionally reset or clear the
>>>> coverage data from one of many threads. Can you say more about what the
>>>> bigger picture is here? There might be an easier way to get it done.
>>>>
>>>> --Ned.
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Chris Withers <chris at simplistix.co.uk>
>>>> To: Ned Batchelder <ned at nedbatchelder.com>, "
>>>> testing-in-python at lists.idyll.org" <testing-in-python at lists.idyll.org>
>>>> Cc:
>>>> Date: Sun, 16 Oct 2016 19:44:38 +0100
>>>> Subject: Re: [TIP] are the absolute paths in .coverage necessary?
>>>> Hi Ned,
>>>>
>>>> Sorry, I meant: do the paths recorded in the .coverage files need to be
>>>> absolute or could Coverage be changed to just record the path relatively to
>>>> the current working directory at the start of execution?
>>>>
>>>> cheers,
>>>>
>>>> Chris
>>>>
>>>>
>>>> On 12/10/2016 22:31, Ned Batchelder wrote:
>>>>
>>>>> Chris, as far as I can tell, they do not have to be absolute. As you
>>>>> can
>>>>> see from the sample of my rc file, you can replace an arbitrary prefix
>>>>> with *, and it will work just fine. How about this:
>>>>>
>>>>> [paths]
>>>>> mylib =
>>>>> mylib
>>>>> */workspace/PYTHON/*/mylib
>>>>> tests =
>>>>> tests
>>>>> */workspace/PYTHON/*/tests
>>>>>
>>>>> --Ned.
>>>>>
>>>>>
>>>>> On 10/12/16 10:59 AM, Chris Withers wrote:
>>>>>
>>>>>> What I currently have is:
>>>>>>
>>>>>> [paths]
>>>>>> mylib =
>>>>>> mylib
>>>>>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/mylib
>>>>>> tests =
>>>>>> tests
>>>>>> /data/jenkins/jobs/mylib-test/workspace/PYTHON/*/tests
>>>>>>
>>>>>> How can I remove the absolute bits of that?
>>>>>>
>>>>>> But, back to my original question: why are the paths in .coverage
>>>>>> absolute in the first place? :-)
>>>>>>
>>>>>> Chris
>>>>>>
>>>>>> On 12/10/2016 15:53, Ned Batchelder wrote:
>>>>>>
>>>>>>> Chris, can you show the absolute paths you are using? I have this
>>>>>>> in my
>>>>>>> rc file to deal with varieties of CI:
>>>>>>>
>>>>>>> [paths]
>>>>>>> source =
>>>>>>> .
>>>>>>> *\coverage\trunk
>>>>>>> */coverage/trunk
>>>>>>> *\coveragepy
>>>>>>>
>>>>>>> --Ned.
>>>>>>>
>>>>>>>
>>>>>>> On 10/12/16 10:00 AM, Chris Withers wrote:
>>>>>>>
>>>>>>>> Hi Ned,
>>>>>>>>
>>>>>>>> Are the absolute paths necessary in .coverage?
>>>>>>>>
>>>>>>>> The reason I ask is that if, for example, they were relative to the
>>>>>>>> cwd, then I wouldn't need a .coveragerc with a whole bunch of paths
>>>>>>>> predicated on the exact filesystem layout of my CI server.
>>>>>>>>
>>>>>>>> thoughts?
>>>>>>>>
>>>>>>>> Chris
>>>>>>>>
>>>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> testing-in-python mailing list
>>>> testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>>
>>> ---------- Forwarded message ---------- From: James <bjlockie at lockie.ca>
>>> To: Bruno Oliveira <nicoddemus at gmail.com>,
>>> testing-in-python at lists.idyll.org Cc: Date: Mon, 17 Oct 2016 09:00:39
>>> -0400 Subject: Re: [TIP] pytest
>>>
>>> The output was: a = <testscript.TestAclasss instance at 0x7f3ca6bf83b0>
>>> def test_foo(a): > assert a == '1' E assert
>>> <testscript.TestAclasss instance at 0x7f3ca6bf83b0> == '1' testscript.py:6:
>>> AssertionError 1 failed in 0.03 seconds
>>>
>>> The missing "self" fixed it.
>>> On 2016-10-16 12:56 AM, Bruno Oliveira wrote:
>>>
>>> Hi James,
>>>
>>> What error do you get? From your example it is missing a self argument,
>>> otherwise it should work just fine.
>>>
>>> Cheers, Bruno.
>>>
>>> On Sat, Oct 15, 2016 at 11:03 AM James <bjlockie at lockie.ca> wrote:
>>>
>>>> I tried to make it work with a class:
>>>>
>>>> import pytest # testscript.py class TestAclasss:
>>>>
>>>> def test_foo(a): assert a == '1'
>>>> Do I need to do something in the __init__ function?
>>>> On 2016-10-14 02:50 PM, Bruno Oliveira wrote:
>>>>
>>>> Hi James,
>>>>
>>>> Thanks for posting the complete example.
>>>>
>>>> There are some problems here, both general Python problems and some
>>>> related to how you are using pytest:
>>>>
>>>> 1. testscript.py has no access to the A function because it doesn’t
>>>> import it.
>>>> 2. You probably want to declare your fixture in a conftest.py file
>>>> so pytest can make it available for all tests in your test suite.
>>>> 3. You have to first declare your option using the pytest_addoption
>>>> hook.
>>>> 4. Since you intend to provide the value passed in the command line
>>>> to the tests using a fixture, it is simpler to skip the test in the fixture
>>>> itself.
>>>>
>>>> Here’s the full example with my suggestions:
>>>>
>>>> # conftest.pyimport pytest
>>>> @pytest.fixture(scope='module')def a(request):
>>>> value = request.config.getoption("A")
>>>> if not value:
>>>> pytest.skip('test needs -A option to run')
>>>> return value
>>>> def pytest_addoption(parser):
>>>> parser.addoption("--A", action="store", default=None, help="a option")
>>>> # testscript.pydef test_foo(a):
>>>> assert a == '1'
>>>>
>>>> Running it:
>>>>
>>>> $ py.test testscript.py -q
>>>> s
>>>> 1 skipped in 0.01 seconds
>>>>
>>>> $ py.test testscript.py -q --A 1
>>>> .
>>>> 1 passed in 0.01 seconds
>>>>
>>>> $ py.test testscript.py -q --A 2
>>>> F
>>>> ================================== FAILURES ===================================
>>>> __________________________________ test_foo ___________________________________
>>>>
>>>> a = '2'
>>>>
>>>> def test_foo(a):
>>>> > assert a == '1'
>>>> E assert '2' == '1'
>>>> E - 2
>>>> E + 1
>>>>
>>>> testscript.py:6: AssertionError
>>>> 1 failed in 0.07 seconds
>>>>
>>>> Hope that helps, Bruno.
>>>>
>>>> On Fri, Oct 14, 2016 at 3:07 PM James <bjlockie at lockie.ca> wrote:
>>>>
>>>> Here is the exact code: $ cat testconfig.py import pytest
>>>> @pytest.fixture(scope='module') def A(request): return
>>>> request.config.getoption("--A")
>>>> $ cat testscript.py import pytest
>>>> skip_A = pytest.mark.skipif( A(pytest.config.request),
>>>> reason="need --A option to run" )
>>>> $ py.test testscript.py ============================= test session
>>>> starts ============================== platform linux2 -- Python 2.7.12,
>>>> pytest-2.8.7, py-1.4.31, pluggy-0.3.1 rootdir: /home/rjl/pytest, inifile:
>>>> collected 0 items / 1 errors ====================================
>>>> ERRORS ==================================== ________________________
>>>> ERROR collecting testscript.py ________________________ testscript.py:4: in
>>>> <module> A(pytest.config.request),
>>>> E NameError: name 'A' is not defined
>>>> =========================== 1 error in 0.01 seconds
>>>> ============================ Instead of doing pytest.config.getoption
>>>> more than one like this: skip_A = pytest.mark.skipif( pytest.config.
>>>> getoption("--A"), reason="need --A option to run" ) I want to do it
>>>> once in the config file. I might want to use the value of the argument
>>>> --A elsewhere so I don't want to do getoption many times.
>>>> On 2016-10-14 01:16 PM, Bruno Oliveira wrote:
>>>>
>>>> Hi,
>>>>
>>>> After fixing the missing “pytest” import, I get this error:
>>>>
>>>> foo.py:8: in <module>
>>>> A(pytest.config.request),
>>>> E AttributeError: 'Config' object has no attribute 'request'
>>>>
>>>> Please make sure to post a complete example and state clearly what your
>>>> problem is and what you are trying to accomplish.
>>>>
>>>> Cheers, Bruno.
>>>>
>>>> On Fri, Oct 14, 2016 at 2:03 PM James <bjlockie at lockie.ca> wrote:
>>>>
>>>> I have this in my config file: @pytest.fixture(scope='module') def
>>>> A(request): return request.config.getoption("--A") skip_A =
>>>> pytest.mark.skipif( A(pytest.config.request), reason="need --A
>>>> option to run" ) but when I run it, it says E NameError: name 'A' is not
>>>> defined
>>>> _______________________________________________ testing-in-python
>>>> mailing list testing-in-python at lists.idyll.org
>>>> http://lists.idyll.org/listinfo/testing-in-python
>>>>
>>>> _______________________________________________ testing-in-python
>>> mailing list testing-in-python at lists.idyll.org
>>> http://lists.idyll.org/listinfo/testing-in-python
>>
>> _______________________________________________
>> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>>
>> _______________________________________________ testing-in-python
>> mailing list testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>
> _______________________________________________
> testing-in-python mailing listtesting-in-python at lists.idyll.orghttp://lists.idyll.org/listinfo/testing-in-python
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20161018/4249bbdb/attachment-0001.htm>
More information about the testing-in-python
mailing list