[TIP] nosetests with coverage excluding a package

[meme dough]

Hi,

Each sub process that starts will need to activate coverage.

The nosetests process will run with coverage enabled for that process
only.  It's not clear what other processes you have.

The ignition.py file starts how many processes?  Did you mean that you
have 1 server process (paster process) and the nosetests process and
only the nosetests process is collecting?

In any event the ignition.py file will need to ensure that any sub
process it creates will activate coverage.

As a shameless plug nose-cov measures sub processes automatically
(provided env vars make it to the sub process).  It does this by
ensuring that any python sub process activates coverage collection
during site initialisation.

If you made the paster process into a test fixture then it would be
handled automatically by nose-cov.  So the test fixture would start
the server before running the tests, then after the tests finish
shutdown the server, then nosetests finishes collecting all coverage
results from all sub processes, merging and outputting any reports you
want.

The advantage of making the paster server process a test fixture is
that any tests that need it can easily just use the fixture, plus you
don't need the extra top level scripts since you can just run
nosetests and it will ensure that you have the appropriate test
resources (the paster process) that the tests require.  Finally since
you run the paster process as a sub process to the nosetests process
then the coverage is automatically activated and collected.

If you keep the paster process started by that wrapper script that
starts both paster and nosetests then you will have to manually
activate coverage for the paster sub process.  You will need to look
at the link Ned provided to do that manually.

Unless you have good reasons for keeping the test wrapper script that
starts the paster server and nosetests and then invokes coverage to
produce an xml report then I'd suggest using a test fixture.  This
seems a better solution to me since you get to use straight nosetests
on the command line and any tests that run that need the server can
have it started / stopped automatically for them (which is basically
what test fixtures are for)!  Then running is simply:

nosetests --with-cov --cov . --cov-report=xml tests/

Presuming you want to measure coverage for all python files under .
(current dir) and tests are located under tests dir and you want an
xml report.

Also it means that when running tests without coverage you can again
just run nosetests straight and it will take care of starting the
paster process for you.

Have a look at http://pypi.python.org/pypi/nose-cov

:)



On 20 July 2011 14:22, Joesan Gabaldon <joesan.gabaldon at gmail.com> wrote:
> We have a script that launches both servers using Popen but only one is
> measured.  So the sequence of the commands for the testsuite are:
>
>     python ignition.py --start-with-pylonsserver
>     nosetests -v thisproject/tests_folder --with-coverage --with-xunit
> --xunit-file=./nosetests.xml
>     coverage xml --include=thisproject/* --omit=./thisproject/tests_folder/*
>     python ignition.py --kill
>
> I saw that subprocess page but didn't look into it much since that one
> server is getting checked.  I'll look into it in the morning though, along
> with the debug command.  Are there any conditions where coverage would
> ignore certain folders/packages by default?
>
> (Ned sorry for the double email, forgot to hit "reply all")
>
> - Joesan
>
> On Tue, Jul 19, 2011 at 10:22 PM, Ned Batchelder <ned at nedbatchelder.com>
> wrote:
>>
>> To get coverage measurement of subprocesses, you need to take some extra
>> steps, unless I'm missing something you're doing in the description below.
>> Are both servers started with Popen?  And one is measured and the other is
>> not?  You can check on measurement with the "coverage debug data" command,
>> which will show you a summary of the data collected during the last run.
>>
>> There's a page in the docs about getting subprocesses measured:
>> http://nedbatchelder.com/code/coverage/subprocess.html
>>
>> --Ned.
>>
>> On 7/19/2011 6:25 PM, Joesan Gabaldon wrote:
>>
>> Hey all, I've been trying to figure this out on and off for a couple
>> months now (it hasn't really been priority until recently) and I could
>> really use some help fixing this.
>>
>> I'm running a system which requires 2 serves to run, one of them being a
>> pylons project.  For some reason coverage doesn't seem to be checking any of
>> the files in the pylons server.  I have a script which launches both servers
>> using Popen, with the pylons server launched with the shell command "paster
>> serve development.ini".
>>
>> Nosetests & coverage are launched with:
>>
>>     nosetests -v thisproject/tests_folder --with-coverage --with-xunit
>> --xunit-file=./nosetests.xml
>>     coverage xml --include=thisproject/*
>> --omit=./thisproject/tests_folder/*
>>
>> I've also tried explicitly telling the coverage report to include the
>> pylons folder which is located at "thisproject/clients/pylons_server":
>>
>>     coverage xml --include=thisproject/*,thisproject/clients/pylons_server
>> --omit=./thisproject/tests_folder/*
>>
>> But that doesn't seem to work, I suspect because that folder is never even
>> executed when nosetests is running the tests.  The functions are definitely
>> getting executed though as those tests pass, and other files in the clients
>> folder and subdirectories of clients are getting included in the coverage
>> report.  Anyone have any ideas?
>>
>> Joesan
>>
>> _______________________________________________
>> testing-in-python mailing list
>> testing-in-python at lists.idyll.org
>> http://lists.idyll.org/listinfo/testing-in-python
>
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
>