[TIP] Test isolation // Detection of offending test
andres.riancho at gmail.com
Wed Dec 5 08:50:36 PST 2012
I've got a project with 500+ tests, and during the last month or
so I started to notice that some of my tests run perfectly if I run
them directly (nosetests --config=nose.cfg
core/data/url/tests/test_xurllib.py) but fail when I run them together
with all the other tests (nosetests --config=nose.cfg core/).
I've tried nosetest's --with-isolation without much luck: it
breaks more than it fixes.
Given that the project has 500+ tests, it is very difficult for me
to identify exactly which one is affecting the tests that fail.
I suspect that this is a common issue when testing, how do you
guys solve this? Is there a way to detect which plugin is not properly
isolated? I thought about writing a script that would do something
for test in all_tests:
result = run_nosetests(test, test_that_fails)
if result == False:
print test, 'breaks', test_that_fails
Is that a good idea?
PS: Sorry for the cross-post between the lists but I thought it was
important for both
Project Leader at w3af - http://w3af.org/
Web Application Attack and Audit Framework
More information about the testing-in-python