[TIP] writing a functional test framework using py.test or proboscis

[Pella,Chris]

Hi all,

We are tasked with writing a test framework which integrates tools written in other languages, and are using python as the glue.  Much of this can be done by wrapping the external tools in either pexpect,  or subprocess. Right now there isn't time to write python bindings for our c or java libraries but that is the preferred way to go in the long run since management wants things happening yesterday (we have virtually no automated testing now).  We are testing systems with a combination of embedded hardware which could be on a local bus or network-attached, with clients that communicate with them either through a library api or various management interfaces, which could be cli-based or have a rest api.

My issue is that this isn't the normal scenario for using unittest-type frameworks, since there are many so external dependancies.  I liked the way I could organize the test dependencies in proboscis and got something very clean-looking up in minutes. e.g. the first test loads a new firmware build if available and subsequent tests would either run or not run based on that pre-requisite. However, I couldn't get any nose plugins to work ( it is built on top of nose) and would like at least have  xunit output  to send to Jenkins at some point, or at least transform into html for reporting.  I would really like to use this tool but I get the feeling I'm going to have to do some poking around to make it generate the output I need and I'm not sure I can succeed since I'm not sure I have the skill-level at this point.

We are also trying py.test, but I'm having trouble figuring out how to deal with external dependancies in a clean way.  The person who is doing this work is using some global variables to set a condition and then skipping tests based on that, and I'm really not happy with that. I'm wondering if there is somebody who has dealt with more complex scenarios using py.test or some other framework and can offer an example of how they dealt with it.

Also, does anybody have any experience using Jenkins to drive distributed functional tests in python? Our builds engineers are reluctant to move to Jenkins because they have this Stygian stable of make files, perl scripts, batch files etc... driven by a php front-end to run builds. I have an idea that by adopting Jenkins to drive test automation it may help convince them to adopt it and we could have some integration between builds and tests , which we don't have now. I doubt we could have Continuous Integration running easily because of the complexity of builds but it would be a step in the right direction.

Chris


The information contained in this electronic mail transmission 
may be privileged and confidential, and therefore, protected 
from disclosure. If you have received this communication in 
error, please notify us immediately by replying to this 
message and deleting it from your computer without copying 
or disclosing it.