[TIP] {Disarmed} Re: Testing multi-threaded applications

[Raphael Marvie]

On May 9, 2007, at 6:54 PM, Bernhard Mulder wrote:
> The disadvantage of this approach might be that the signaling of  
> the event is maybe only needed for testing, and is not necessary  
> for production code. I don't know the testing literature, so I  
> don't know if it says something about such testing artifacts. I  
> personally value fast and reliable tests very highly, so I would  
> just leave this artifact in the code.

Sometime we use methods onyl for testing purposes. As long as the  
overhead is not too much when generating the event compared to your  
"normal" processing it does not hurt me. There should be an  
evaluation of the ratio noise vs benefit.

> Maybe I should mention that my own testing code is not working yet,  
> since I turned my attention away from the threading aspects of the  
> application to deal with other areas first.

But keep in mind that you are working in a multi-threaded context,  
otherwise you may have trouble when introducing your threads. It is  
the same problem with distributed systems (which is my background):  
taking a non distributed application and inserting support for  
distribution has never lead to a good application.

> One of the things I want to test is some sort of timeout mechanism:  
> the application hands off some task to some separate process, and  
> needs some fall back mechanism in case this process dies or is  
> inaccessible (it might run on another machine). I will use the  
> time.sleep function in the production code, but for testing, I will  
> run my application on "simulated time" rather than real time, so  
> that, again, the tests can run at full speed. One question I have  
> is: how do I model this? I could replace the function sleep in the  
> time module for testing, but I am afraid that changing standard  
> functions might interfere with the testing framework.

When I have a threaded object that need to sleep, I pass explicitely  
the sleep function as a parameter with defaulting to the one I  
commonly use in production. Thus, no dependencies are hidden in the  
thread and I can easily switch it for the tests. Something like:

class Worker(threading.Thread):
     def __init__(self, sleep=time.sleep):
         self.sleep = sleep
     def run(self):
         # ...
         self.sleep(waiting)
         # ...

With such a class, the test can go:

class TestWorker(unittest.TestCase):
     def sleep(self):
         time.sleep(short_waiting) # or nothing...
     def test_worker(self):
         w = Worker(self.sleep)
         w.start()
         # ...

If you do not want to externalize the sleep function, at least  
externalize the waiting.


> My current (planned, since the tests are not quite functional yet)  
> solution is to introduce an indirection: import time.sleep into  
> some module called SystemInterfaces, and then import sleep from  
> SystemInterfaces.sleep. This way, my testing scaffolding can  
> substitute a custom sleep function for the standard one.

The indirection is not necessary from my point of view

> Similar issues arise with the interface to the file system.
>
> Is there some other recommended way to deal with this issue?

To some point, externalizing the opening of files.

HTH

r.

--
Raphael Marvie, PhD                http://www.lifl.fr/~marvie/
Maître de Conférences / Associate Professor  @  LIFL -- IRCICA
Directeur du Master Informatique Professionnel spécialité IAGL
Head of Master's in Software Engineering     +33 3 28 77 85 83