[TIP] Functional testing of asynchronous systems

[Kumar McMillan]

On Sun, Nov 14, 2010 at 4:17 PM, Joseph Heck <heckj at mac.com> wrote:
> Hey all,
>
> I've been googling around this weekend, looking for what may already exist out there that (more effectively) support functionally testing of asynchronous systems.
>
> To date, I've been doing hacky/stupid things like embedding "sleep(...)" into my code to wait for a sufficient time to verify completion, but this has all sorts of obvious drawbacks. I've also been mocking in systems with fake interfaces to be able to unit test the logic constructs effectively - with some success. But what I really want to do is functionally test the whole system.

Twisted trial has some builtin utilities for async testing
http://twistedmatrix.com/trac/wiki/TwistedTrial

However, I don't think you necessarily need a framework for this kind
of testing.  I'd suggest avoiding sleep/assert though because they
will be slower on fast machines than they need to be.  Instead, I'd
suggest a waiting loop where you check repeatedly for the condition.
Something like:

done = False
while not done:
    # use a counter to implement a timeout here
    if check_condition():
        done = True
    else:
        time.sleep(0.5)

>
> I'd love to take advantage of a unittest like framework with assertions as well as JUnit style XML output so that I can measure/graph the results of the testing over time. My intention is to clean-slate some systems with a nightly build through BuildBot or Hudson and then drive the functional testing from that same system.
>
> Finally, I'd love to enable something like simple webhooks in this testing tool so that if I'm writing the asynchronous systems, I can optionally call back into this thing to register the timing/performance of the system and track that over time as well. Maybe that's just better held completely outside any testing mechanism, but getting a verification.
>
> If folks are doing this kind of thing today, is it always with their own custom code? Any particular libraries of components that help and/or make this easier?
>
> -joe
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>