[TIP] Coverage does not cover script if py.test executes it from another directory

[Ned Batchelder]

On 9/2/13 11:45 AM, Christoph Buchner wrote:
> Hi folks,
>
> I have run into an issue using py.test, pytest-cov and coverage. The
> question is also on Stackoverflow in all detail
> (http://stackoverflow.com/q/18573542/599884), but I'll also
> summarize it here:
>
> I'm writing a python script which takes command line arguments, working with
> some files (git repos etc).
> I've written succeeding tests with py.test putting this script through its
> paces (verifying it works correctly with different arguments, etc),
> executing it with subprocess.call in the tests.
>
> Now I want to analyze code coverage with coverage.py.
> Coverage, when used via the pytest-cov plugin (which has subprocess-handling
> already built-in/ootb), does not see/cover my script when the script is
> called from a temporary testing directory created with py.test's tmpdir
> fixture.
> Coverage does see my script when it's called in the directory it resides in
> (and the filename argument points to a remote path).
>
> To illustrate with a minimal example,
>> Running in directory /home/cbuchner/pytest_experiment
>> Command: ./my_script.py /tmp/pytest-52/test_10/arg_file.txt
> gets correct coverage measurement, but
>> Running in directory /tmp/pytest-52/test_11
>> Command: /home/cbuchner/pytest_experiment/my_script.py arg_file.txt
> gives "Coverage.py warning: No data was collected."
> See http://stackoverflow.com/q/18573542/599884 for full
> console output and link to the used code.
>
> In both situations, my tests pass! This is on Ubuntu, with Coverage 3.6,
> pytest-2.3.5, pytest-cov 1.6, all from PyPi.
>
> Questions:
> * How can I get coverage to recognize my script even if it's executed in
> another directory?
> * Is this a bug in coverage, or something which is just not possible to do?
> Would be surprised if the latter, after all, tmpdir is a stock mechanism of
> py.test, right?
> * If the way I did it is an inappropriate testing implementation, I'd be
> grateful for pointers how to "properly" coverage-test file-manipulating
> console utilities written in Python. Writing automated tests is pretty new
> to me, so I'm learning much and know little (yet).
>
> thanks for any pointers on this,
> best,
> Christoph
>
It's a little hard to know why it's not working without more details, 
but a first thing that comes to mind is that coverage ignores code it 
thinks is in the stdlib.  If your script is being run from a directory 
under the stdlib somehow, then that could explain it.

Can you run "coverage debug sys" in the same context as your failing 
scenario?  It will show details of the configuration.

--Ned.