[TIP] coverage.py and pyramid web framework?

Ben Cohen cohen.ben at gmail.com
Tue Aug 28 13:55:57 PDT 2012

To follow up on this issue --

I switched to using the wsgi server included in the cherrypy framework instead of the waitress wsgi server for my application.  With this server I see coverage of the code from all my views when I run my test suite -- all of which are not marked as covered when I use the same test suite with the waitress wsgi server.

I'm guessing there is a difference in the way these two servers implement threading that causes this to occur …?  I haven't delved into either server implementation but from my logs I see these thread names from cherrypy:

[CP Server Thread-6]

versus thread names of this variety from waitress:


Its not entirely clear to me -- but I wonder if waitress perhaps spawns threads in response to requests rather than using a pre-initialized worker thread pool.  Would that explain what I'm seeing if it was true?

I thought perhaps I was mistaken about what coverage should do in the case of a multi-threaded application -- so I conducted an experiment with the cherrypy server -- 2 requests were generated which were handled by two different worker threads, and then a third worker thread was responsible for generating the coverage report (via a call to the html_report method of the coverage api object).  This produced the hoped for coverage report showing statement coverage along all the paths taken by each thread -- along the expected path for the view method calls.

So I'm left with a little bit of confidence in the observation that coverage appears to be working when I use the cherrypy wsgi server but not when I use the waitress server.

Many thanks for your help.  Let me know if there's anything I can do to help clarify this or if you think it sounds like I'm doing something 'inherently wrong' which will be likely to bite me down the road …


On Aug 24, 2012, at 9:21 AM, Ben Cohen <cohen.ben at gmail.com> wrote:

> Thanks for all the responses so far!
> For completeness heres the project link for venusian decorators:
> http://pypi.python.org/pypi/venusian
> Venusian style decorators add attributes to function/methods/classes including a callback which is invoked by a later 'scan' stage -- the pyramid framework uses this system to mark function/methods as web request handlers in a nice/flexible fashion.  But Tres's links have convinced me that there's nothing inherently 'magic' about this process which would lead to coverage.py missing function/method calls.
> I tried running coverage conventionally like so:
> coverage run /path/to/my/pythonenv/bin/pserve development.ini
> pserve is a script that's part of the pyramid web framework.  The high level view of what it does is:
> 1. create the wsgi object representing a web application
> 2. setup a wsgi server 
> 3. pass the application object to the server for serving.  
> After starting the server thusly, then invoking my test suite, I kill the serve process and run:
> coverage html
> to generate the html report.  The report is essentially the same as the one generated by the api calls and shows that none of my application's view methods are being called...
> I had mentioned earlier that I was using the single-threaded waitress http server -- however that's not true -- waitress is a multi-threaded server -- I'm using it in its default configuration.  There's a little bit of magic with threads happening largely behind the scenes within the framework/http server -- my functions/methods will all be called in a thread from a thread worker pool -- would it be expected for coverage to fail to trace methods called in a thread other than the MainThread?
> On Aug 24, 2012, at 7:48 AM, Tres Seaver <tseaver at palladion.com> wrote:
>> Hash: SHA1
>> On 08/24/2012 10:35 AM, Tres Seaver wrote:
>>> On 08/23/2012 08:34 PM, Ben Cohen wrote:
>>>> The most obvious shared feature of the functions/methods that
>>>> report no coverage is the use of a venusian callback method
>>>> decorator -- this decorator marks functions/methods with declarative
>>>> attributes, subsequently modules are scanned for methods with those
>>>> attributes and then ingested by the pyramid web framework's
>>>> web-publishing machinery.
>>>> Is there some magic happening within the pyramid framework that 
>>>> causes coverage to fail to register when these methods are called?
>>>> Is anybody using coverage and pyramid together without issue …?  I
>>>> didn't expect so fundamental an issue with the two as the pyramid 
>>>> documentation has a chapter describing the use of coverage in unit 
>>>> tests …
>>> Pyramid itself runs tests with the coverage package without issues
>>> (run via 'setup.py nosetests --with-xunit --with-xcoverage' -- see the
>>> 'cover' environment in its tox.ini).  We use it routinely for apps
>>> built on Pyramid as well.  The venusian decorator does not do anything
>>> to interfere with coverage:  the attributes it adds to the decorated 
>>> function are used to populate Pyramid's registry during the later
>>> "scan" phase.
>> Pyramid's Jenkins:
>> http://jenkins.pylonsproject.org/job/pyramid/
>> and the instance for SubstanceD, an app written on top of Pyramide:
>> http://jenkins.pylonsproject.org/job/substanced/
>> Tres.
>> - -- 
>> ===================================================================
>> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
>> Palladion Software   "Excellence by Design"    http://palladion.com
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>> iEYEARECAAYFAlA3lB8ACgkQ+gerLs4ltQ4tmQCgx9jQjZPXJJlme1ZrNqm46rMb
>> KEIAn2SE/Cz43KS/f+61XVb6HcqtiHzK
>> =vlKO
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20120828/15ee6561/attachment.htm>

More information about the testing-in-python mailing list