[TIP] Test parameterisation: TestCase.subTest, testscenarios (was: Supplying variable options to unittest2 tests)

Robert Collins robertc at robertcollins.net
Mon Jun 27 17:26:55 PDT 2016


On 28 June 2016 at 12:15, Ben Finney <ben+python at benfinney.id.au> wrote:
> Robert Collins <robertc at robertcollins.net> writes:
>
>> Sounds like you have a test parameterisation problem. the
>> 'testscenarios' library[1] would be a decent fit for this - I wrote it
>> to tackle problems such as the one you're having, and it is a unittest
>> compatible library so should drop straight into your framework.
>> [1] https://pypi.python.org/pypi/testscenarios
>
> I am an enthusiastic user of ‘testscenarios’, it is a great way
> toconcisely express data-driven tests that exercise the same logic on a
> set of inputs.
>
> In Python 3.4, the ‘unittest.TestCase’ API grew a “subtest” feature
> <URL:https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests>
> which seems similar in intent but is rather different in operation.
>
> so now we have the very capable ‘testscenarios’ third-party library, and
> the rather different standard library ‘TestCase.subTest’ API.
>
> What are people's thoughts on which to use and when? Are they
> complementary, or is one clearly superior?

I perceive subtests as more lightweight, suitable for smaller numbers
of dimensions, and for less systematically-applied-parameterisations.

E.g. if you have 40 test cases to parameterise, subtests would require
you to write a for loop within each one, whereas testscenarios doesn't
require that - but does require you to name each scenario.

-Rob



More information about the testing-in-python mailing list