[TIP] Randomizing test order with nose
Scott David Daniels
Scott.Daniels at Acm.Org
Wed Apr 22 09:51:40 PDT 2009
Charles McCreary wrote:
> About two years ago, Titus had a "random idea" in which he proposed a
> nose plugin that would
> randomize the test order. In one of the projects I'm involved in, that
> has turned out to be rather
> important. I can do this within the unittest framework with:
>
> class TestXMLHTTPResponder(unittest.TestCase):
> ... tests...
>
> if __name__ == "__main__":
> import random
> suite =
> unittest.TestLoader().loadTestsFromTestCase(TestXMLHTTPResponder)
> random.shuffle(suite._tests)
> unittest.TextTestRunner(verbosity=2).run(suite)
>
> But I use nose, so a plugin that would randomize the order would be
> helpful. Before I charge off and write one, has anyone already done it?
Be careful here. For randomized tests in any form (including execution
order), it is
vital to be able to rerun _exactly_the_same_test(s)_. So, provide a
command line
arg (or whatever) that allows you to seed the random number generator,
and always
provide the seed used when running a randomized system.
--Scott David Daniels
Scott.Daniels at Acm.Org
More information about the testing-in-python
mailing list