[TIP] Test a system with paramatrized state, parametrized function input, and results.

[Stuart Doherty]

Hi All,
  I've been chatting with Ronnie and Holger about this on irc and it was suggested I bring the issue up on the list.  I'm trying to evaluate py.test for our integration and system test needs.  We're developing a fairly substantial QA lab to test the mobile network equipment we're making... but nevermind all that, we can distill things down to abstract software components for now.

So, I have a system under test (SUT).  The SUT can be initialized/kicked-off with some set of config values.  The state of the system can then be modified through a function call.  After the state has been modified there are different functions we can call to probe the system to ask it about its state.  I want to be able to conduct a collection of tests where the input to each individual test run is:

1.  The initialization/config of the system
2.  The argument(s) to the function(s) that change state on the system.
3.  The values expected to be returned from the probe functions. 

I know I could do this kind of thing when I was doing some Django development.  I was able to inheret from unittest.TestCase, and then any fixture loading I did at the class level was only done once.  It would be this kind of behaviour I think I want, except the fixture to load, and the values I verify against, woudl be input to the test runner.

I put an example of a system:

http://paste.pocoo.org/show/207741/

And how the py.test test module might look:

http://paste.pocoo.org/show/207740/

Neither would actually work in py.test.  Its just to give you a sense for what I'm trying to do.

Does anybody have any thoughts on this?

Stu

P.S. If you want some more detail on our actual application, here goes:
-We make mobile network gear.
-We want to have a regression system setup where we can run a series of network capture files (PCAP format) into a clean system.
-For each PCAP we want to probe different tables on the system database, and verify that the entries are what we expect.
-Clean out system, rinse and repeat for a series of pcap/resultant-value combinations.
-Assertions on incorrect values, etc.
-A subset of tests woudl be used as a pre-svn-commit step for developers.