[TIP] coverage.py and pyramid web framework?
Ben Cohen
cohen.ben at gmail.com
Tue Aug 28 13:55:57 PDT 2012
To follow up on this issue --
I switched to using the wsgi server included in the cherrypy framework instead of the waitress wsgi server for my application. With this server I see coverage of the code from all my views when I run my test suite -- all of which are not marked as covered when I use the same test suite with the waitress wsgi server.
I'm guessing there is a difference in the way these two servers implement threading that causes this to occur …? I haven't delved into either server implementation but from my logs I see these thread names from cherrypy:
[CP Server Thread-6]
versus thread names of this variety from waitress:
[Dummy-3]
Its not entirely clear to me -- but I wonder if waitress perhaps spawns threads in response to requests rather than using a pre-initialized worker thread pool. Would that explain what I'm seeing if it was true?
I thought perhaps I was mistaken about what coverage should do in the case of a multi-threaded application -- so I conducted an experiment with the cherrypy server -- 2 requests were generated which were handled by two different worker threads, and then a third worker thread was responsible for generating the coverage report (via a call to the html_report method of the coverage api object). This produced the hoped for coverage report showing statement coverage along all the paths taken by each thread -- along the expected path for the view method calls.
So I'm left with a little bit of confidence in the observation that coverage appears to be working when I use the cherrypy wsgi server but not when I use the waitress server.
Many thanks for your help. Let me know if there's anything I can do to help clarify this or if you think it sounds like I'm doing something 'inherently wrong' which will be likely to bite me down the road …
Ben
On Aug 24, 2012, at 9:21 AM, Ben Cohen <cohen.ben at gmail.com> wrote:
> Thanks for all the responses so far!
>
> For completeness heres the project link for venusian decorators:
>
> http://pypi.python.org/pypi/venusian
>
> Venusian style decorators add attributes to function/methods/classes including a callback which is invoked by a later 'scan' stage -- the pyramid framework uses this system to mark function/methods as web request handlers in a nice/flexible fashion. But Tres's links have convinced me that there's nothing inherently 'magic' about this process which would lead to coverage.py missing function/method calls.
>
> I tried running coverage conventionally like so:
>
> coverage run /path/to/my/pythonenv/bin/pserve development.ini
>
> pserve is a script that's part of the pyramid web framework. The high level view of what it does is:
>
> 1. create the wsgi object representing a web application
> 2. setup a wsgi server
> 3. pass the application object to the server for serving.
>
> After starting the server thusly, then invoking my test suite, I kill the serve process and run:
>
> coverage html
>
> to generate the html report. The report is essentially the same as the one generated by the api calls and shows that none of my application's view methods are being called...
>
> I had mentioned earlier that I was using the single-threaded waitress http server -- however that's not true -- waitress is a multi-threaded server -- I'm using it in its default configuration. There's a little bit of magic with threads happening largely behind the scenes within the framework/http server -- my functions/methods will all be called in a thread from a thread worker pool -- would it be expected for coverage to fail to trace methods called in a thread other than the MainThread?
>
> On Aug 24, 2012, at 7:48 AM, Tres Seaver <tseaver at palladion.com> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 08/24/2012 10:35 AM, Tres Seaver wrote:
>>> On 08/23/2012 08:34 PM, Ben Cohen wrote:
>>>
>>>> The most obvious shared feature of the functions/methods that
>>>> report no coverage is the use of a venusian callback method
>>>> decorator -- this decorator marks functions/methods with declarative
>>>> attributes, subsequently modules are scanned for methods with those
>>>> attributes and then ingested by the pyramid web framework's
>>>> web-publishing machinery.
>>>
>>>> Is there some magic happening within the pyramid framework that
>>>> causes coverage to fail to register when these methods are called?
>>>> Is anybody using coverage and pyramid together without issue …? I
>>>> didn't expect so fundamental an issue with the two as the pyramid
>>>> documentation has a chapter describing the use of coverage in unit
>>>> tests …
>>>
>>> Pyramid itself runs tests with the coverage package without issues
>>> (run via 'setup.py nosetests --with-xunit --with-xcoverage' -- see the
>>> 'cover' environment in its tox.ini). We use it routinely for apps
>>> built on Pyramid as well. The venusian decorator does not do anything
>>> to interfere with coverage: the attributes it adds to the decorated
>>> function are used to populate Pyramid's registry during the later
>>> "scan" phase.
>>
>> Pyramid's Jenkins:
>>
>> http://jenkins.pylonsproject.org/job/pyramid/
>>
>> and the instance for SubstanceD, an app written on top of Pyramide:
>>
>> http://jenkins.pylonsproject.org/job/substanced/
>>
>>
>> Tres.
>> - --
>> ===================================================================
>> Tres Seaver +1 540-429-0999 tseaver at palladion.com
>> Palladion Software "Excellence by Design" http://palladion.com
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iEYEARECAAYFAlA3lB8ACgkQ+gerLs4ltQ4tmQCgx9jQjZPXJJlme1ZrNqm46rMb
>> KEIAn2SE/Cz43KS/f+61XVb6HcqtiHzK
>> =vlKO
>> -----END PGP SIGNATURE-----
>>
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120828/15ee6561/attachment.htm>
More information about the testing-in-python
mailing list