[TIP] Randomizing test order with nose

Scott David Daniels Scott.Daniels at Acm.Org
Wed Apr 22 09:51:40 PDT 2009


Charles McCreary wrote:
> About two years ago, Titus had a "random idea" in which he proposed a 
> nose plugin that would
> randomize the test order. In one of the projects I'm involved in, that 
> has turned out to be rather
> important. I can do this within the unittest framework with:
>
> class TestXMLHTTPResponder(unittest.TestCase):
> ... tests...
>
> if __name__ == "__main__":
>     import random
>     suite = 
> unittest.TestLoader().loadTestsFromTestCase(TestXMLHTTPResponder)
>     random.shuffle(suite._tests)
>     unittest.TextTestRunner(verbosity=2).run(suite)
>
> But I use nose, so a plugin that would randomize the order would be 
> helpful. Before I charge off and write one, has anyone already done it?
Be careful here.  For randomized tests in any form (including execution 
order), it is
vital to be able to rerun _exactly_the_same_test(s)_.  So, provide a 
command line
arg (or whatever) that allows you to seed the random number generator, 
and always
provide the seed used when running a randomized system.

--Scott David Daniels
Scott.Daniels at Acm.Org




More information about the testing-in-python mailing list