[TIP] nose programmatically & subset of tests?

[Ringo De Smet]

Hello,

I am using nose inside another Python application where I expose test
functionality. I first experimented with the nosetests command line
script to find out about the various arguments. I wants to find all
tests in a certain folder (not the current working directory) and run
either all or a subset of the tests. With nosetests, I was able to
perform this with:

all tests: nosetests --where /opt/qbase3/var/tests --verbosity 4
subset of tests: nosetests --where /opt/qbase3/var/tests --verbosity 4
volume_driver

Taking the same set of options and passing that on to nose.run() as
the argv list doesn't give me the same behaviour:

Python 2.6.2 Stackless 3.1b3 060516 (release26-maint, Oct 26 2009, 15:34:55)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import nose
>>> nose.run(argv = ['--where','/opt/qbase3/var/tests','--verbosity','3'])
volume_driver.TestBasicFunctionality.testRead ... ok
volume_driver.TestBasicFunctionality.testWrite ... ok
volume_driver.TestBasicFunctionality.test_evens(0, 0) ... ok
volume_driver.TestBasicFunctionality.test_evens(1, 3) ... FAIL
volume_driver.TestBasicFunctionality.test_evens(2, 6) ... ok
volume_driver.TestBasicFunctionality.test_evens(3, 9) ... FAIL
volume_driver.TestBasicFunctionality.test_evens(4, 12) ... ok
TestTopLevel.TestClassLevel.testClassMethod ... ok
TestTopLevel.testTopLevel1 ... ok

======================================================================
FAIL: volume_driver.TestBasicFunctionality.test_evens(1, 3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qbase3/lib/python/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/opt/qbase3/var/tests/volume_driver/TestBasicFunctionality.py",
line 13, in check_even
    assert n % 2 == 0 or nn % 2 == 0
AssertionError

======================================================================
FAIL: volume_driver.TestBasicFunctionality.test_evens(3, 9)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qbase3/lib/python/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/opt/qbase3/var/tests/volume_driver/TestBasicFunctionality.py",
line 13, in check_even
    assert n % 2 == 0 or nn % 2 == 0
AssertionError

----------------------------------------------------------------------
Ran 9 tests in 0.012s

FAILED (failures=2)

Running all tests is OK, but when I try to take a subset of these
tests, I get this:

>>> nose.run(argv = ['--where','/opt/qbase3/var/tests','--verbosity','3', 'volume_driver'])
volume_driver.TestBasicFunctionality.testRead ... ok
volume_driver.TestBasicFunctionality.testWrite ... ok
volume_driver.TestBasicFunctionality.test_evens(0, 0) ... ok
volume_driver.TestBasicFunctionality.test_evens(1, 3) ... FAIL
volume_driver.TestBasicFunctionality.test_evens(2, 6) ... ok
volume_driver.TestBasicFunctionality.test_evens(3, 9) ... FAIL
volume_driver.TestBasicFunctionality.test_evens(4, 12) ... ok
TestTopLevel.TestClassLevel.testClassMethod ... ok
TestTopLevel.testTopLevel1 ... ok
volume_driver.TestBasicFunctionality.testRead ... ok
volume_driver.TestBasicFunctionality.testWrite ... ok
volume_driver.TestBasicFunctionality.test_evens(0, 0) ... ok
volume_driver.TestBasicFunctionality.test_evens(1, 3) ... FAIL
volume_driver.TestBasicFunctionality.test_evens(2, 6) ... ok
volume_driver.TestBasicFunctionality.test_evens(3, 9) ... FAIL
volume_driver.TestBasicFunctionality.test_evens(4, 12) ... ok

======================================================================
FAIL: volume_driver.TestBasicFunctionality.test_evens(1, 3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qbase3/lib/python/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/opt/qbase3/var/tests/volume_driver/TestBasicFunctionality.py",
line 13, in check_even
    assert n % 2 == 0 or nn % 2 == 0
AssertionError

======================================================================
FAIL: volume_driver.TestBasicFunctionality.test_evens(3, 9)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qbase3/lib/python/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/opt/qbase3/var/tests/volume_driver/TestBasicFunctionality.py",
line 13, in check_even
    assert n % 2 == 0 or nn % 2 == 0
AssertionError

======================================================================
FAIL: volume_driver.TestBasicFunctionality.test_evens(1, 3)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qbase3/lib/python/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/opt/qbase3/var/tests/volume_driver/TestBasicFunctionality.py",
line 13, in check_even
    assert n % 2 == 0 or nn % 2 == 0
AssertionError

======================================================================
FAIL: volume_driver.TestBasicFunctionality.test_evens(3, 9)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/qbase3/lib/python/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/opt/qbase3/var/tests/volume_driver/TestBasicFunctionality.py",
line 13, in check_even
    assert n % 2 == 0 or nn % 2 == 0
AssertionError

----------------------------------------------------------------------
Ran 16 tests in 0.022s

FAILED (failures=4)

Hmm, first all tests are run, followed by the selected subset. On top
of that, if I execute the subset immediately after starting the Python
interpreter, the result is even different. In this case, first all
tests are run, followed by an ImportError with message "No module
named volume_driver".

So I'm puzzled on how to use nose programmatically, and use nose
functionality multiple times in the same Python session. The docs are
quite terse on this matter. My aim is to get my requirements working
that I mentioned at the beginning: run all or a subset of the tests
from a certain folder not being the current working folder.

Hope someone can help out here.

Ringo