[TIP] paramterized or generative tests [was nose2: the nosening]

Robert Collins robertc at robertcollins.net
Thu Aug 5 20:15:51 PDT 2010

On Fri, Aug 6, 2010 at 2:54 PM, Victoria G. Laidler <laidler at stsci.edu> wrote:
> Hi Michael & all,
> I desperately need a clean way to run generated tests that a) get names I can control and b) attributes that can be defined and modified in the test itself that are then attached to the test case so I can access them via my report generator plugin.

testtools + testscenarios will do what you need.

> I tried to do this with nose and it has really not worked out well - we are still using it in some places but just gave up and wrote a couple of simpler Pandokia-compliant test runners. I'd rather use a community standard so if we could get this into nose2 that would be great.
> My immediate use case is as follows:
> - I have a directory full of datafiles that define the inputs and expected outputs for the code under test
> - The test itself should be python code that can:
>   - import bits of test code (not just code under test) from elsewhere, including the code that does the following:
>   - dynamically generate the tests based on the files that are present in the directory

the load_tests or test_suite hooks permit this step - load_tests is in
python2.7 and presumably unittest2 - you can use unittest2 with the
testtols TestCase to get both this discovery and the
TestCase.addDetails API

>   - name each test based on those filenames

scenarios = [(filename1, {attribute:value, attribute2:value2}), .(..), ...]
return parameterise_tests(scenarios, basic_tests)

>   - invoke and execute the code under test

The stock run step will then do that.

>   - perform comparisons and save data about the comparisons in dictionaries attached to the test function

you could call addDetail(name, info) in setUp; however if you are only
interested in failed comparisons, you could use a Matcher - Matchers
support get_details in their contract from testtools 0.9.4 (I think),
and that will automatically attach back to the result object. In stock
TestResult objects this is reported as a string; using a testtools
enhanced TestResult its reported as a dict of content objects so you
can tunnel json or whatever rich data you want through it.


More information about the testing-in-python mailing list