[TIP] Randomizing test order with nose

[Douglas Philips]

On or about 2009 Apr 23, at 9:25 AM, Michael Foord indited:
> What I was actually suggesting didn't involve the use of a PRNG,  
> although that might not work for every situation.

True.

> Even if a PRNG is used to generate the randomized order, if you  
> record the order a PRNG need not be involved to reproduce it if an  
> alternative runner can get the precise order from the logs of the  
> previous run.

Right now we don't record the actual random values returned because it  
would completely swamp the human-readability of our logs. Once we  
started attacking this problem we discovered that the minor  
variability in device timings caused some of our tests to consume  
varying amounts of randomness. That pretty much forced us to re-seed  
before each test to make sure we could re-run them with the same  
results, individually, some tests just won't ever re-run the same way  
twice because of the device timing variability, so we're stuck with  
isolating the effects of that to an individual test. But I suspect  
this is specific enough to our testing environment to be of little  
general interest. This is one place where subclassing TestCase and  
providing a setUp that re-seeds is the easy answer. :)

-Doug