[TIP] adding custom tests to a unittest test run

[holger krekel]

On Thu, May 17, 2012 at 00:57 -0700, Chris Jerdonek wrote:
> On Thu, May 17, 2012 at 12:00 AM, holger krekel <holger at merlinux.eu> wrote:
> >> > On Tue, May 15, 2012 at 09:25:28PM -0700, Chris Jerdonek wrote:
> >> >> I often have the need to add extra tests to my unittest.main() test
> >> >> runs.  The tests I need to add are tests that depend on data obtained
> >> >> at run-time, e.g. user-provided command-line options.
> >
> > I think it's generally a better idea to skip tests when a configuration
> > or command line option is missing rather than to not collect them.
> > IOW, it's a good idea to keep the overall number of tests stable.
> 
> Yes, I see the value in keeping the number of tests stable if that
> were possible.
> 
> However, one of the use cases, for example, is building a suite of
> test cases dynamically from an external directory of test data (e.g. a
> directory of files containing lists of input and expected output).
> The runtime data to provide would be the path to the directory, which
> may or may not be available.  Without the directory, you have no
> choice but not to collect those tests, so there would be no tests to
> skip.

You are right, of course.  For data-driven tests the collect-then-skip
technique does not work or make much sense.  I've often seen the case
where people invoke test functions multiple times with different data
which depends on command line or environment options.

best,
holger