[TIP] General purpose testing frameworks

[Noufal Ibrahim]

Hello everyone,

   I'm working for a company where we're setting up infrastructure to aid
addition/running of tests for the software developed. The first requirement
was a CI framework. I've deployed buildbot successfully and it's doing it's
job fine. The second (and more vague) requirement is a for a general purpose
test framework. We're developing some hardware and will have some tests that
run in special conditions, some which run like 'normal' unit tests etc. None
of these are in Python. Most of them are system level tests (protocol
compliance, interoperability with 3rd party tools etc.). I need to come up
with something that can select/run tests in a uniform way. It would be a
real productivity killer to have to remember multiple tools for different
kinds of tests. I guess what I'm looking for is a test harness where the
accouting/book keeping part is decoupled from the actual test runners. I can
extend the harness then by writing runners that would handle the different
kinds of tests.

   II've been doing some googling and have found subunit which can, as far
as I can tell, decouple the test results from the harness. If I can convert
my tests to use this, I can write a single harness to interpret the subunit
results and produce a report. It's a 'component' rather than a full fledged
test harness though.

   I've also found pandokia which is interesting and I'm trying it out right
now.

   I went through the wiki page on test tools in python and found qmtest
which I've just downloaded today. I am trying it out.

   The requirement though seems quite natural and I get the feeling that
there's something obvious that I'm overlooking. Most of the searches I do
give me tools that are specific to 'unit testing' python applications which
is not what I'm trying to do.

   I'd appreciate any recommendations/suggestions anyone has on such
frameworks which I might be able to use/extend.

Thanks.

-- 
~noufal
http://nibrahim.net.in
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idyll.org/pipermail/testing-in-python/attachments/20091221/834d151f/attachment.htm>